Height is an issue, but more so, the lack of parallax left to right. The perspective will never match up correctly except for a viewer who aligns their eyes with where the camera is when you render. This is what OSL shaders do in the 3d world, but that won't translate to playback on an actual plasma screen in an installation. Not sure there is a live solution without the user wearing some kind of glasses/viewer.
You can test this - render a box interior and full screen it on your monitor and then move around. It can still look nice, but it won't look "real". Closest thing I can think of is Beeple's Human One, but it's not a direct comparison.
This exact problem is what Unreal engine solves on volume stages. The software has position of the camera lens at all times and it modifies/moves/adjusts the content to that lens perspective and makes the parallax effect work.
You can get away with it on large billboards with lots of distance, but it becomes harder the closer the user is to the display. I don't think you're missing anything - at the end of the day, your user is still looking at a 2d plane that is representing a static 3d image.