This a very tricky question. You have many valid points. As I don't know where you are from, I can't make any factual observation but I can tell you that you are on the right track in the complaint department.
As you did not purchase anything at the cafe, in most cases they are perfectly justified in not allowing you to use the restrooms. I did not say they were correct, just possibly in the right in a legal sense.
I would go about as follows, depending on what the laws for slander are in your area. Write a not too nasty letter to the people in charge of the event. Explain it like you did above, and mention the name of the cafe. Put at the bottom of the letter that you will be sending a copy of the letter to the local newspaper if you do not get an answer from them with "X" number of days.
If you do not hear from them, call and ask if they got the letter. If you get stonewalled, send the next one return receipt requested explaining that this is the second one you have sent.
While you are doing this, contact the manager of the cafe. (Unless he/she was there at the time.) If the manager was not there, explain to him/her exactly what happened. If the manager does not come up with an acceptable response then you start to tell everyone you know, by word of mouth only, put nothing in writing, what vile creatures are working there.
If the event council still does not respond, try to find a good bottom feeding lawyer who will be willing to sue for emotional damage, for a percentage, with no upfront costs. Try to put everything in writing EXACTLY as it occured. Be very careful with your notes since, if you sound "unhinged", and the notes are called forth in a court of law, people may take the other side. Reasonable and prudent go a long way with a jury or judge.
By the way. Complaining to the cafe about no public toilets is a waste of time. As far as I know, there is no law that requires an establishment like a cafe (Unlike large stores such as supermarkets, etc.) to have public rest rooms.
Unfortunately, today, many people use what little power thay have in hateful, ways and just out of spite. It makes them feel good. I certainly hope that if you can't at least open their eyes, you can hit them right in the wallet. My condolences to your children. I hope you get satisfaction from this. It is truly terrible to deny a child since they have no comprehension as to why the "bad thing" is happening.
Good luck to you.