How to stand up straight while skating forward?

This may sound weird, but when ever I skate while playing hockey, I always end up skating with my foot bending inwards a centimer or two. Like, I don't skate on the blade, I skate on the side of it. It's hard to explain but I find it hard to skate forward while standing straight :/