Version 1.4.0 EARLY ACCESS (with working DLSS!) (Patreon)

So, I have good news and bad news. Which do you want first? Oh yes, I already went and spoiled some of the good news in the post title: the mod is working again, having fixed the compatibility problems with the latest 1.0.1436.25 build of RDR2, and not just that: DLSS is now fully usable in VR! Of course, if you had downgraded the game to keep playing during the last few days, you'll need to let it update again to get the new features. The bad news is that, due in part to the added complexity introduced by DLSS and in part to new annoying game bugs introduced by Rockstar, the mod has become a little harder to set up, and it requires you to jump through a few more hoops in order to have the best VR experience.

Buckle up, because this is going to be a long post :-) I will organize it in numbered paragraphs, to make it easier to refer to portions of it in the comments, and I expect to add/revise some of the paragraphs according to your feedback. As usual, the download and the README with the quick setup instructions are at the bottom of the post: however, make sure to read the rest of the post before firing up the mod, otherwise chances are that you will have an unsatisfactory VR session.

Before delving into the technical configuration details, I'd like to share another piece of good news: my mods now work correctly with the new Vive Pro 2 headset []-) No more jitters in the right eye! Amazingly, Patron Dustin Silver was able to pester HTC support into passing on a message to their developers team, and to quote his words, "Lo and behold, they fixed it!" Patch notes here. "Fixed a bug where flicker is observed on the right-eye when playing content using R.E.A.L mod". That made my day!!!

• 1. Game bugs. Since the update to build 1.0.1436.25, RDR2 no longer honors the graphics settings stored in its system.xml file reliably. In particular, on my PC at least, every time the game is launched it always seems to reset TAA to High (which is good, as Medium has haloing artefacts) and Resolution Scale to Off, which is VERY bad, because my mod used to rely on a Resolution Scale greater than 1 to produce the high-resolution renders needed for the VR headset without requiring absurdly large resolutions on the monitor.
  This was the main reason for my delay in releasing 1.4.0. Since the game now seems to always start up with Resolution Scale = Off (or 1/1), and VR does need high values of Resolution Scale, I had to revise large sections of the code to make sure that the mod could tolerate changing graphics settings on the fly without crashing or starting to behave erratically. In the end, it was a win, as you will now be able to experiment much more easily with the settings (no more need to hand-edit system.xml and/or to restart the game at every tweak), but it did take a lot of effort and time.
  Some users on Reddit are complaining of similar problems where the game "forgets" the settings for Anisotropic Filtering, Soft Shadows and so on, but for the moment I haven't been able to reproduce them, whereas on my system the Resolution Scale bug is pretty consistent.
  
• 2. Graphics settings presets. If I had to choose the most common complaint that I heard from Patrons and YouTubers alike about the mod graphical quality, it would be without a doubt "too blurry / low resolution". For this reason, and also due to the above mentioned Resolution Scale bug that requires a special countermeasure, I decided to unify the resolution across all graphics presets. Now they all provide the same resolution as the Ultra preset, i.e., a monitor window size of 1080x1080 and a Resolution Scale of 5/2,  translating to an effective VR resolution of 2700x2700 pixels. If that is too much for your video card and it kills your frame rate, you can either choose a lower quality preset (worse graphics but keeping the same high resolution) or lower the Resolution Scale from the game menus and the mod overlay (read on for further details).
  As the Very High and Ultra preset used to differ only in Resolution Scale, Very High was eliminated. And since I found out that quite a few people like to use the optimized settings provided by YouTube channel Hardware Unboxed, I added them as a new preset selectable from RealConfig.bat. Credit goes to HUB of course.
  
• 3. New mod options in the overlay. In order to work around the game bugs, to display new important information, and to provide DLSS-related options, I had to add a few controls to the mod UI. The "R.E.A.L. VR" overlay tab was becoming a little crowded, so I decided to split it and create a game-specific tab. Under this new "RDR2" tab, you will find an information line detailing the Resolution Scale and DLSS quality that the game is using at any given moment.
  Remember that DLSS, which employs an NVIDIA-proprietary neural network to upsample the images rendered by the game, and Resolution Scale, which uses an internal rendering buffer and a TAA implementation coded by Rockstar, are mutually exclusive. The game logic forbids using both together, and it will automatically force-disable Resolution Scale if DLSS is selected.
  You will find new info in the "Game fps" line too: besides the frame rate and latency, I now show the current VR resolution, which is fundamental to keep track of because it's the resolution of the buffer that actually gets sent to your VR headset. To make an example: if your game window on the monitor is 1200x1200, and you have Resolution Scale currently set to 2/1, the images needed for VR will in fact be extracted from a buffer that is 2400x2400 pixels large. That is the resolution that counts for the mod: not what the game shows in its graphics settings page, nor the one you might find in your headset control panel.
  The "Advanced" section in the "RDR2" tab lists the old options that you already knew, to fix TAA for Stereo VR, change the world size to play with action figures or giants, and compensate the camera rotation for smooth turning. It also holds new controls that I will discuss in the following paragraphs.
  
• 4. Fighting the Resolution Scale bug. Unless you want to use DLSS instead of the normal rendering pipeline of the game, in which case you should see the next paragraphs, Resolution Scale must be restored to the correct value every time the game is started. You can of course do it manually each time you boot the game: enter the settings menu, verify that NVIDIA DLSS is Off, restore the Resolution Scale setting to your preference, then confirm. If you have the same bug as me, the preference will be saved to your system.xml file, but the next time you run the game it will be forgotten again.
  So, I added a quality-of-life improvement to the mod that can force the game to behave. At the bottom of the "Advanced" section in the "RDR2" tab, you'll find a new drop-down named Resolution Scale override. If this is set to any value except "No override", the mod will automatically reset the game render pipeline each time you start RDR2, and during the reset it will force the game to use the Resolution scale of your choice. Note this means that the corresponding setting read from the system.xml file will become irrelevant, so the place where you want to select your preferred Resolution Scale is now the drop-down menu and not the settings file itself.
  
• 5. DLSS (at last!). This is certainly the most anticipated section of this whole post, but unfortunately due to Rockstar's choices it's also the one I fear will cause the most head-scratching.
  DLSS works very differently from Resolution Scale (and as I mentioned earlier, they cannot be used together).
  With Resolution Scale, the game actually renders the world to an internal buffer which can be smaller or larger than the output window, and then rescales this buffer (respectively with an upsampling or downsampling operation), optionally after using TAA to produce very high-quality anti-aliased images.
  VR needs ultra-high resolution images (3000x3000 pixels is not uncommon), but very few users have 8K monitors that could display such a huge window. So, I leverage the fact that with Resolution Scale > 1 the game actually renders to an internal, larger buffer, and have the mod send the contents of that buffer to the VR headset, instead of sending the final output window. With the Ultra preset for instance, that allows for a VR resolution of 2700x2700 while keeping the window size on the monitor to a quite manageable 1080x1080 pixels.
  DLSS doesn't work that way. It only ever upsamples. The game always renders to a smaller buffer than the output window, and then the neural network upsamples the image to its final size. For example, with NVIDIA DLSS set to "Performance", the game renders to a 540x540 buffer and the network interpolates that image to 1080x1080. Of course, such small images would be unsuitable for VR. So, the idea to take advantage of DLSS in a VR mod is to have a huge output window, let's say the usual 2700x2700 pixels, make the game render to a reduced version of that image (e.g., 1800x1800 with DLSS set to "Quality"), and then send to the VR headset not the lower resolution render but the final upscaled output image.
  [If you are following me closely, you might have noticed that DLSS has an intrinsic problem wrt to the Resolution Scale approach. With DLSS, the high-resolution detail is always faked. The neural network can be incredibly good, but it's still making it up as it goes. With Resolution Scale instead, all the pixels sent to the VR headset were actually drawn by the game, with full quality. It's of course a trade-off between fidelity and performance.]
  Now, the problem is that RDR2 natively won't ever allow an output window that is larger than the monitor on which it is running. There are tricks to exploit in the video card driver like DSR or custom resolutions, but my experience has always been they are too complex and unstable for the average user. Also, RDR2 doesn't seem to like them very much either.
  So, to make DLSS possible with the R.E.A.L. VR mod for RDR2, I had to force the game's hand and make it accept any large window size independently of what your monitor resolution is. To the best of my knowledge, that cannot be done just altering the system.xml file or the game command line, so I added specific code and exposed the setting in the "Advanced" tab under the name "Base window size override".
  
  Here is what you need to do if you want to use DLSS in VR with good results. Go to the game Settings->Graphics menu, and select a preset for DLSS from Auto, Quality, Balanced, Performance, Ultra Performance. [For the number-curious, Quality is 2/3 resolution (e.g., render at 720x720 and upsample to 1080x1080), Balanced is 29/50 (don't ask, I have no idea why they chose this crazy ratio), Performance is 1/2, Ultra Performance is 1/3 (so a tiny 360x360 render to show a 1080x1080 window).] When you select the DLSS preset, you will notice that Resolution Scale becomes grayed out in the game settings and is no longer selectable. Confirm and exit the settings menu. In the mod overlay, you should also set "Resolution Scale override" to "No override" to avoid confusion later.
  Now (essential step) choose from the mod's "Base window size override" drop-down the actual resolution that you will want to use for VR, for example 2700x2700, and press the "Resize window" button that just appeared under the drop-down. There is always a very slight chance that resizing the window might crash the game, so don't do this after playing for a lot of time without saving :-)
  At this point, if everything went well, you should see under the "Game fps" line that Resolution Scale is Off, DLSS is set to the quality preset you selected, and the VR resolution is exactly the one you chose from the drop-down list.
  You won't need to repeat these steps every time you run the game: luckily the game remembers its DLSS settings (because they're the new kid in town and everybody loves them), and the mod will automatically apply the base window size override at every launch.
  Now, there is a significant issue with this approach: if your monitor is not large enough to contain the full window, the game will play just fine (thanks to my little magic ^_^) and everything will be visible in VR, but on the monitor some portions of the window will be offscreen, which means invisible and most importantly unreachable with the mouse. This will make navigating the game menus and the mod overlay a big nuisance, so I strongly advise using a gamepad for DLSS high-resolution VR gaming.
  
  Finally, my preliminary evaluations of the technique: my benchmarks show around 15% performance boost vs Resolution Scale (for the same VR resolution of course), so nowhere around the 45% figure that all the press quoted uncritically. In general you can expect higher gains in the wilderness, and less advantage in towns. "Ultra Performance" is good only for the laughs: it's basically unwatchable in VR where everything is magnified and large as life. "Quality" on the other hand is pretty good, almost looking like native, except for issues like invisible particles, birds leaving black trails in the air, checkerboard hair and the such, as can already be seen in several YouTube videos.
  All in all, I still prefer Resolution Scale, which is a little more performance intensive but much more solid in terms of uncompromising quality. Some people say it's too blurry even at high resolutions, but that can easily be fixed using the TAA Sharpening slider. I set it to halfway in my presets, but even pushing it all the way to the right doesn't create objectionable artefacts, and it makes for a very sharp image. Fun fact: many people on the Internet swear by CAS (Contrast Adaptive Sharpening) and say that the TAA Sharpening implementation by Rockstar is vastly inferior. During my testing as I was developing this release, I hotswapped Rockstar's shader with the one for CAS, and tested the full ranges of parameter values: there were no visible differences between the two filters, at any level of magnification. So basically Rockstar have CAS integrated into their TAA implementation, and yet everybody hates it and says that it's blurry while CAS applied via ReShade is the bee's knees. Oh well.
  
• 6. DLSS for VR. During these days of hammering the new game update to make it fit with the mod (and vice versa) I discovered a couple of interesting things. The first was quite saddening at first: in stereoscopic VR, DLSS suffered from the same kind of artefacts and haloes around nearby objects and characters that plagued TAA, only they were much worse and made the reconstruction almost insufferable. Not NVIDIA's or Rockstar's fault, to be sure: they simply didn't account for the fact that RDR2 would be played in 3D.
  As luck would have it, most of the code I had already written to fix TAA could be adapted quite easily to also fix DLSS, and even more providentially, during the past week NVIDIA released their DLSS SDK to the public, which made me more confident that the solution I was pursuing was the right one. So, the "Fix TAA for Stereo" checkbox became "Fix TAA/DLSS for Stereo".
  The second interesting thing was that Rockstar implemented the DLSS sharpening function that NVIDIA provides (and which has the side effect of increasing frame time a little bit, so not exactly zero-cost), but for some reason they didn't expose it to users. It is in fact a very subtle effect, so perhaps they feared a barrage of negative comments from people saying things like "Rockstar botched their sharpening implementation as usual!". Anyway, I exposed the additional setting in the mod, so that we can enjoy a little placebo effect by confirming that our DLSS is as sharp as it can possibly be ;-)
  
• 7. README. I've updated the README in some points, but for now I'm not including all of this new information in there, since Rockstar will in all likelihood come out with further updates in the near future. Bugs might be corrected, or new ones might be introduced. Until the dust settles down, I prefer to keep the README down to a manageable size and to use posts such as this one to discuss all these new options and countermeasures.
  Incidentally, for those of you who are new to the modding world, this kind of breaking/restoring of mods when a game gets important updates that fundamentally change its rendering pipeline is quite common. However, it isn't usually as bad as this time. For instance, GTA V just got updated, and the only intervention needed to restore full functionality for my VR mod was for users to download an updated version of Script Hook V. Zero effort on my side. So, it varies: thank you for your patience and support during these past few days.
  

TO BE CONTINUED. Phew. That's all for today. I'm exhausted, and if you read all of this you must be too. Let me know how this new release works for you, and have fun tinkering with DLSS and the new options! Don't forget to play the game a bit too, as I'm no longer finding any time for it :-D

CHANGELOG

v1.4.0:

• made the mod compatible with RDR2 build 1.0.1436.25
• added workarounds for new game bugs
• reorganized the mod overlay and added new info and controls
• added full support for DLSS and window sizes larger than the monitor resolution
• completely fixed DLSS for Stereo rendering
• made the mod robust to adjustments to the rendering pipeline at runtime (like settings changes or window resizes)
• implemented settings and version detection code
• added automatic control of the overlay font to adapt to dynamic changes in the window size
• reorganized the graphics preset files to favor resolution and added a popular config

v1.3.1 and older:

• see this post for previous changes