That is very curious. Some aliasing is expected in RealFlight by default because, as another user said, it does not have any antialiasing built in. You must enable it via your Nvidia control panel.
I wondered if it was being applied at all, but the plane in the vehicle editor's preview pane looked properly smoothed.
There is an interesting telltale sign you can look for within RealFlight. When AA is enabled and taking effect, the Gadget Tabs menu on the left side of the screen will have gaps between the icons.
I suggest checking that just to confirm it matches expectations when you think AA should be on and working.
It seems like something else is going on here, though. I suspect Windows display scaling is a factor.
For reference, the attached image shows what I see in windowed mode on a 1920x1200 monitor. I switched to manual zoom mode and zoomed all the way in at Eli Field trying to replicate an image you posted. On top is no AA, the bottom is with several AA settings forced on and cranked up via Nvidia's CP.