While we’ve seen Lots of examples of AI that have made games look better over the past few yearsI’ve never seen an approach like this possible Grand Theft Auto V. and by the power of magic makes it look so damn real.
This is a project by Stephan R. Richter, Hassan Abu AlHaija and Vladlen Koltun at Cornell University. culminates in a newspaper called Improvement of photo realism. Both the paper and the accompanying video contain quite a few technical details. Here is the basic summary of their activities:
We present an approach to improve the realism of synthetic images. The images are enhanced by a convolution network that uses intermediate representations produced by traditional rendering pipelines. The network is trained on a novel controversial target that provides strong surveillance at multiple levels of perception. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We believe that this is one of the causes of strong artifacts that can be observed in the results of many previous methods. To remedy this, we propose a new strategy for scanning image fields during exercise. We are also introducing several architectural improvements in the deep network modules that are used to improve photorealism. We confirm the advantages of our contributions in controlled experiments and report significant gains in stability and reality compared to newer image-to-image translation methods and a variety of other baselines.
In short, machines can only work with what they have been told. By improving the quality of the feed and the way it is fed, the results can be dramatically improved. Like this:
If you really want to get into it, even more than in the video above, you can read the entire paper Here.