Researchers from the University of Surrey have pioneered an innovative method, utilizing the popular video game Grand Theft Auto V, to enhance the accuracy of generating 3D models from dog photographs. This award-winning study showcases the potential applications of artificial intelligence (AI) in predicting the 3D pose of dogs, with implications ranging from wildlife conservation to virtual world creation.
The research team, led by Moira Shooter, a Postgraduate Research Student, trained an AI system on images of CGI dogs within the virtual environment of Grand Theft Auto V. Despite being trained on computer-generated canines, the model demonstrated its adaptability by generating precise 3D skeletal models from real photographs of dogs. Shooter highlighted the versatility of the model, suggesting its potential for aiding conservationists in identifying injured wildlife or assisting artists in crafting more realistic animals within virtual spaces.
Traditionally, teaching AI systems to derive 3D information from 2D images involves providing them with “ground truth” data, detailing the actual 3D positions of objects. In human terms, this often requires motion capture suits, a logistical challenge when working with animals.
To overcome this hurdle, the researchers turned to virtual simulations. By modifying the code of Grand Theft Auto V through a process known as “modding,” the team replaced the main character with eight different types of dogs. They created a diverse dataset, named DigiDogs, consisting of 118 videos showcasing virtual dogs engaged in various activities under different weather and lighting conditions, totaling 27,900 frames.
The next phase involves refining this dataset using Meta’s DINOv2 model, ensuring its ability to accurately predict a 3D pose comparable to real dog photographs.
Shooter emphasized the richness of information contained in 3D poses, asserting that this innovative solution holds promise across diverse fields, from ecology to animation. The study opens up possibilities for leveraging AI technology in novel ways, showcasing the transformative potential of virtual environments in training and enhancing AI systems.
