Please think of the pressure point of your sail (kinda near the center a few feet above the foot) Imagine it similar to the Center of Gravity on a diagram
If the sail is fully extended, the Center of Sail Pressure is back a few feet from the mast, and as you Reef or Roll Up the sail it moves forward as the sail shrinks
Imagine having the sail fully extended with the main sheet pulled and locked back tight then imagine the wind coming from over the middle of the kayak.
The wind is perpendicular to the sail and all of it's force is on the Center of Pressure (just above forward seat)
If you configure it this way, on the water, the Sail (and Kayak attached to it!) will want to pivot around the Mast into the wind - the rudder is the only thing to prevent this pivoting.
On a sailboat with a tiller rudder, you have to pull the tiller to the windward side constantly (weather helm) turning the bow away from the wind to hold the boat on it's course instead of pivoting into the wind ( bow coming Up into the wind) if you had released the tiller. Our steering is opposite of a tiller but the concept is the same, you are constantly fighting the sail or center of pressure as it wants the balance the forces on the sail and kayak.
As others mentioned, releasing the main sheet (Line w/ clip attached to the back corner of the sail) will allow the center of pressure to shift forward as the sail flops in the wind, or Reefing the sail (rolling it up) moves the sail Center of Pressure forwards, reducing the Weather Helm effect. Also sitting in the aft seat (if solo) helps reduce Weather Helm by using more of the hull under water farther aft to hold the kayak from pivoting around the mast!
Also, pedalling thru a Tack , keeps the water speed over the rudder, giving it more strength to turn or hold the bow and finally, holding the sail a bit longer on the old, downwind side can help pivot the boat to the new Tack as the wind crosses over the bow and now pushes against the sail, adding momentum to the bow as it moves to the new course (as it gets close, then release the sail to the new downwind side)
The opposite is true for Jibe-ing (wind crossing over the stern during a turn) if the sail is perpendicular to the kayak or course on downwind then call out "Jibe Ho" to warn your passengers that the sail will smack them in the heads if they don't duck! As you turn the stern thru the wind, you have to tighten the main sheet and sail to the center of the kayak before then allowing the sail to gradually swing thru the wind to the opposite side of the kayak and then release the sail main sheet back out if your course remains downwind. You can pull the sail over by hand if the winds are light from one side to the other during a Jibe.
On large sailboats, a metal boom or pole holds and controls the foot of a sail, so having the winds YANK it across the stern during a Jibe has Killed Sailors by crushing their skulls, so develop a good habit for yourself and passengers while Jibe-ing by calling it out and controlling the sail!