Taking good-quality photographs can be a challenging task, as it typically requires finding ideal locations, angles and lighting conditions. Although artistic pictures have so far primarily been taken by human photographers, in recent years, some researchers have started investigating the possibility of taking pictures automatically using robots.

To this end, a team of researchers at Purdue University and Adobe Research has recently developed a new framework that allows a robot to automatically capture photographs of humans, specifically portraits. This framework, presented in a paper prepublished on arXiv, makes robots follow a human user to an ideal spot and then take a picture of them.

LeRop, the framework developed by the researchers, was specifically designed to take indoor portraits of human subjects. First, the framework guides a robot towards a favorable or desired location for taking pictures and then it uses a photo evaluation model to propose the best views, as well as deep reinforcement learning (DRL) model to adjust the robot's position and orientation to ensure the best lighting conditions.

"When composing is activated, the robot attempts to adjust its position to form the view that can best match the given template image and finally takes a photograph," the researchers wrote in their paper. "A template image can be predicted dynamically using an off-the-shelf photo evaluation model by the framework, or selected manually from a predefined set by the user."

LeRop is an interactive framework, as users can program it to follow one target to the spot where she wishes to capture a photograph. Once the user reaches that spot, the robot starts searching for the best view to capture. LeRop's DRL component is what ultimately allows it to adjust its viewpoint based on how it matches template images.

The researchers decided to equip the robot with a 360-degree camera and a high-quality main camera, as this allows it to have a full view of its surroundings at all times without continuously rotating or switching to different viewpoints. Interestingly, the framework has a modular structure, which means that all its models can be replaced or adapted based on a user's needs.

The team evaluated LeRop in a variety of trial tests on three indoor scenes, integrating it within a simple robot called Turtlebot. The robot took 20 photographs of a human user in each of these three settings, 10 with a predefined template and 10 using templates that it generated dynamically.

The average number of adjustments made by the robot was 11.20 for predefined templates and 12.76 for dynamically generated ones. In addition, using the researchers' framework, the robot could take pictures in an average time of 22.11 seconds using predefined templates and 22.40 seconds using dynamically generated ones.

The LeRop framework could be a first step towards the creation of an efficient robot photography system that can take high-quality portraits quickly and automatically. However, the system developed by the researchers still has a number of limitations that could prevent it from being adopted on a large scale. For instance, so far, it has limited on-board computation power, and thus only works on a powerful remote computer.

In addition, the researchers have so far only integrated it within Turtlebot, which is a simple robot with very few degrees of freedom. In their next works, they would like to test their system on a more complex robot with more degrees of freedom.

"Our system currently only supports a single-person portrait," the researchers wrote. "New policies would need to be re-trained to get better support on taking group photos. In future work, we also would like to test different photo evaluation aesthetic models and extend the work to outdoor scenes."

---

---

More information: LeRoP: A learning-based modular robot photography framework. arXiv:1911.12470 [cs.RO]. arxiv.org/abs/1911.12470

© 2019 Science X Network