There's no national standard for tree removal permits — requirements are set at the city or county level, and sometimes layered on top by an HOA, so this is general orientation, not a substitute for checking your specific local ordinance.
Municipalities that require a tree removal permit typically trigger it on one or more of: trunk diameter over a set threshold (commonly measured at a standard height above ground, often called "diameter at breast height"), specific protected or "heritage"/"landmark" tree designations, location on or near a public right-of-way or property line, or location within an environmentally protected area (wetland buffers, view corridors, slope-stability zones). A young, small tree in the middle of a private backyard is far less likely to need a permit than an old, large tree near a street or a protected zone — but this varies enough by jurisdiction that it's worth an actual phone call, not an assumption.
Start with your city or county planning department, or the urban forestry division if your municipality has one — many post their tree ordinance and permit application online. If you're inside city limits vs. an unincorporated county area, the answer can differ block to block, so confirm which jurisdiction actually governs your address.
If you're in a homeowners association, its covenants may require separate approval for tree removal — sometimes stricter than the municipal rule, sometimes covering trees a city permit wouldn't touch at all (aesthetic/landscaping rules rather than protection rules). Check your HOA's governing documents or ask the board before removal, independent of whatever the city says.
Jurisdictions with real tree-protection ordinances can levy significant fines for unpermitted removal, sometimes calculated per inch of trunk diameter removed, and some require replacement planting on top of the fine. A reputable local tree company that works in your area regularly will usually know the local requirement and can often pull the permit as part of the job — worth asking about when you get quotes.