Spud put it down to a learning experience but don't ever despair about finding fools gold because where there is pyrite there is usually gold and they are often mixed together. Pan your extra buckets of dirt it will be good practice and you never know you may find yourself a nugget.
Pyrites will generally float on the surface of the water in the pan and will break up in your fingers.
Gold by the way can float its all about the surface tension of the water and the size of the flakes. Its not likely to float because it is 19x heavier than water but it can if the conditions are right. You should search youtube for some videos on Gold Panning techniques. I know one of the ones by Garrett on their Super Sluice pans explains how gold can float and how when floating it can group together.
To stop it floating you add a drop of Finish Dish washing liquid the blue stuff (Jet Dry as the Americans call it) to your pan, this breaks the surface tension of the water allows the gold to drop easily. It only takes a drop so a bottle will last a long time. I also add a drop to my highbanker water when I am recirculating the water for it out of a large plastic container.
Also have you seasoned your gold pan?
Seasoning the gold pan makes the water glide off rather than bead which will happen with a new plastic pan. That beading doesn't help the surface tension of the water issue. To season the pan get some sandpaper or a kitchen scourer and rub the inside of the bottom of the gold pan, this scratches up the surface a bit and allows the water to glide off. There is a youtube video on this also. You can also do this out in the field with a handful of gravel in the pan and just rub it against the bottom of the pan.
The old timers used tin pans and they would season them by throwing them on the coals of a fire and "Blueing the Pan" as it was known because the tin would turn a bluish colour on the inside. They would then sand down a few specific areas on the pan, each prospector had his own method of doing this and it was a closely guarded secret.
Big thing to remember about gold panning is its not the riffles that will get your gold.
The old timers tin pans didn't have riffles in them they were added much later as a precaution and they only added a single small indent ring to theirs near the top of the tin pan around its circumference. The area you need to worry about and keep your eyes on when panning is the inside lip a the very bottom front of the gold pan, its the thing that will catch your gold. It doesnt matter how fancy the design of the gold pan, they all have that in common.
The only exception to this rule are the traditional pans of other countries such as the more common Batea gold pan used in South America its shaped like a bowl with only a very slight angle to it. It uses a different but efficient panning technique all its own. Other traditional pans include the conical Dulang of South-East Asia, Bucket of Kyrgyzstan, the Ninja bowl of Mongolia, the Lotok of Russia and the Rubber mat of Mongolia.
Hope this helps
