Library and books are a GREAT practice---congrats on incorporating that into your children's lives! Reading to your kids is soooooooooooo important, and its so important to spark their intellectual curiosity. It's an easy investment that will pay off over their entire life.
If you live near a park or a spot with some pavement, try bringing some sidewalk chalk and letting them draw on the sidewalk! Kids love anything artsy, and you could also draw a hopscotch pattern they can hop on and get tired out.
Love the water idea. It does tire kids out very effectively. Swimming lessons would be great---I think swimming is an important skill for everyone. My 1.5 and 3.5 yr old nieces are crazy about their little inflatable pool.
If you live someplace green and leafy go on a nature walk and start introducing them to some basic tree and plant identifications. You could bring a few different leaves or flowers home and use them to make artwork, to press between wax paper and make a card for grandma, etc.
If you have a children's museum near you, consider buying a membership. Kids at that age thrive on repetition and could probably go once a week and love it as much every single time, and most museums have daily presentations, activities, etc.