Rice milk can be made with beta amalyze an enzyme derived from barley which is chametz and would make the rice milk chametz.
The following brands of OU Kosher pareve original rice milk do not contain any chametz, but are kitniyot:
Rice Dream Enriched Unsweetened Original
Market Basket Enriched
Price Chopper Enriched
Wild Harvest Enriched