I love that you are mixing things up in your diet, Dave! The biggest differences between coconut and almond milk compared with cow's are sugar and protein.
Cow's milk does not contain added sugars (with the exception of flavored milks), therefore it has lower sugar content when compared with vanilla almond, coconut, or soy milk. However if you are sticking with the plain/unflavored version you should not have any added sugars and even the light versions will be lower in added sugars if you are going for a flavored option. As you mentioned it had a sweet flavor, i imagine you chose the vanilla flavor. There is actually a plain almond milk that is only 35 calories per cup! Though this brings me to my next point...
Although almond milk sounds like it should be high in protein, it's actually quite the opposite. 1 cup of almond milk or coconut milk has a mere 1 gram of protein. Soy milk is far more comparable to cow's milk and is a good source of protein in your diet. But if you are eating plenty of protein in the other foods in your diet (meat, poultry, seafood, nuts, seeds, beans, cheese, yogurt, eggs) as are most Americans, then you have nothing to worry about.
Just make sure that whichever milk you are choosing has been fortified with calcium (at least 25%) and vitamin D. Silk brands are fortified...
Cheers to good health and good food!