Did you ever read The Nanny Diaries? Authors McLaughlin and Kraus make an astute observation. If the mother of the children works full time herself, then she will usually be pretty respectful of you as a working woman, and in that case you can tell her that you can't be available those days. If the mother is a privileged rich lady who wants a servant to watch her children, then maybe that won't work as well for you. Either way, you serve at her pleasure, so the day she decides she is ready to replace you, she will. But, if the novel is any indication, telling her you aren't available weekends may just be the moment she decides to replace you.
I worked as a nanny for 2 years. My employer was the working woman type, and she treated me with respect.