Frivolous Tax Returns The IRS can charge penalties of $5,000 for Frivolous Tax Returns and other Frivolous submissions made to the IRS. The US Tax Court may impose penalties of up to $25,000 when a Taxpayer makes frivolous arguments in Tax Court. A list of some of the frivolous arguments that can lead to penalties