I'm sure you will get as many different opinions as the number of people you ask.  However, I think it boils down to what you believe.  What is best for you and your family?  

You can get opinions and thoughts about the topic, because they might have a different view point or mention something you haven't thought about.  However, it is a much bigger conversation with your family.  Discuss why you want to go back to school.  Ask them how they would feel if you weren't as available to them all the time because you had to go to class or study.  

If you haven't already, make a list of pros and cons.  Be honest.  Sometimes it can be difficult to acknowledge what we really don't want to.

Follow your gut.  If you examine all sides of going back to school, you will make the right decision.  And just because you make a decision, doesn't mean you can't change your mind.  So if you start taking classes and it just becomes too much, then put it on hold until your children are older.

Hope this gives you some food for thought.