There is possible chometz in most baby rice cereals, so the OU cannot recommend them for Passover use. However, there is no problem owning rice and making your own cereal over Passover.

One should make the cereal using a special non-Pesach, non-chametz pot, and this special pot (and any special utensils used for the cereal) should not be washed together with the Pesach dishes. You can Kasher a pot before Passover and dedicate it for making the rice real over Passover.

Similiarly, the grinding of the rice should not be done in a Passover grinder on Passover. You can grind the rice in a clean chometz grinder before Passover or purchase a dedicated grinder for use over Passover.

Sample Recipe:
Rice Cereal
1. 1/4 c. rice powder [brown or white (not enriched) rice ground in blender, food processor, or coffee grinder].
2. 1 cup water.
3. Bring liquid to boil in saucepan. Add the rice powder while stirring constantly.
4. Simmer covered for 10 minutes, mix in formula or breast milk and fruits if desired
5. Serve warm.