If San Marco is the heart of Venice, then Rialto Market is the mouth!
Open for business since the beginning of the last millenium, the Rialto Market is a must-see for all visitors to Venice. The boats of produce and fish arrive to make their deliveries to the various stalls around 5:00 AM and most vendors are in hawking mode by 8:00 AM. You can see chefs from some of Venice's top restaurants picking out the best piovra (squid) and other swimming delicacies alongside Italian nonne grandmothers and tourists in the Pescaria (Fish Market). Closer to the Rialto Bridge are the produce vendors, selling mostly seasonal fruits and vegetables and even local specialties such as artichokes from Sant' Erasmo (island in the Northern Lagoon) and radicchio from nearby Treviso.
By about 1:00 PM, most vendors have packed up for the day so definitely do come in the morning, and the market is open Tuesday - Saturday.
Saturdays at the Rialto Market is a Venetian tradition. Locals and tourists alike come to peruse and shop for fresh fish, vegetables and fruit, possibly visiting the other food specialty shops in the area, but definitely taking part in the ritual of cichetti (Venetian tapas) a glass of prosecco... or two or three from one of the many bars that line the surrounding streets. On sunny days, it becomes the Venetian equivalent to brunch, but here you hop from bar to bar, conversing and getting tipsy with your group of friends the entire way.
While you can't plan for one, you can at least swing by the squares where they are known to occur, especially on the weekends in Venice: Campo Santa Maria Nuova in Cannaregio, the campo outside of the Frari Church in San Polo, Campo San Barnaba in Dorsoduro, Campo San Maurizio in San Marco, Via Garibaldi in Castello or even La Pescaria of the Rialto Market (on Sundays and Mondays when the Fish and Fruit sections are closed).