Yes. “Supposed to.” That doesn’t mean it always happens.
The point is that the church building faces the rising sun which symbolizes Christ and that the direction of East symbolizes God (the two are inter-related of course). On the Last Day, Christ will come “from the East” and so we are supposed to be facing that way, watching for Him.
It’s all symbolic. It’s not a requirement but it is a very strong Christian Tradition.