You have two main options: A handheld GPS, or a fish finder with GPS. If you fish from your kayak, a fish finder-GPS combination is a great way to go. If you don't fish, and just want the GPS for navigation, information about distance and speed, and maps, a handheld GPS would be the way to go.
With our Oasis, no fishing, I use the Garmin Oregon 550 T. It is very accurate, and it has tide tables for basically every tidal water in the Ca N Bay area. The 550 T is probably the easiest to use hand held GPS's my sons and I have ever owned. I put a piece of electrical tape on the plug in sites on the unit to prevent salt water from seeping in. Also, I use the little orange water wings in case it gets dropped overboard. The Oregon 550T is great and easy to use for all types of land situations, walks and treks. The batteries are rechargeable and have lasted a full day in a yak or on land with juice still left.
With the Freedom Hawk fishing yaks I use the Lowrance Elite 4 DSI. If and when I trade up to a single Mirage fishing yak, that will be the unit. The Maps are very accurate, and the underwater "pictures" are excellent. Fish are a little hard to find. I use it mainly to find structure to fish or holes where the fish lay and feed and hide.
I have a decade plus in age Lowrance M68 for our jon boat and a portable transducer for the Freedom Hawks. The GPS maps are a little old but still mostly reliable. It is deadly re finding Stripers. It has never been babied and is in great shape. We take it off when we get home and put it back on when we go fishing. It runs all day on one of the small 12 volt batteries or with the big 12 volt for our Minn Kota Saltwater Bow mount without cutting into use of the Minn Kota.