Canada is so vast, you have to choose a side and focus. As a child, my family went to Toronto after Niagara Falls. 
Others go to Quebec or Montreal. On this trip with our sons, we did Vancouver and Victoria after seeing Washington State and Idaho in the U.S. This was a perfect route since we took our rental car onto the ferry that took us to Vancouver and Victoria.
When we had done enough sightseeing, shopping and eating at the Victoria town plaza, we craved a little adventure in the wilderness and ocean. We found a thrilling zip lining excursion at the Sooke National Forest. I had not intended to zipline and wanted to read a book for the two hours that the boys and my husband would be zipping away.
http://adrenalinezip.com/adventure-packages/
But my family talked me into it—and I’m so glad they did! I’m not comfortable with heights or small spaces and zip lining includes both of those! I had to really coach myself through those scary moments but it really ended up being such a thrilling experience!! We also went whale-watching which is featured on another post on this blog.
http://www.tourismvictoria.com/things-to-do/whale-watching/ 
Stories, descriptions and recommendations on this can be found in the book, A Magic Carpet Ride. Profits from the book go to various charities.
www.amazon.com/dp/069271393X/ref=rdr_ext_tmb
Follow this plan to build trip itineraries with your family: