Squirrel hunting is one of the most affordable ways to get started.

Rifle or shotgun
A .22 rifle works for still targets, and a light shotgun works for moving squirrels, where each is legal.
Comfort
A seat cushion and bug spray make early season sits more pleasant.
Cleaning
Rubber gloves are a good idea when cleaning squirrels.