Academia.edu uses cookies to personalize content, tailor ads and improve the user experience.
By using our site, you agree to our collection of information through the use of cookies.
To learn more, view our Privacy Policy.×
Figure 1: Our context-aware framework performs the assessment and labeling of images in photo col... more Figure 1: Our context-aware framework performs the assessment and labeling of images in photo collections by considering image quality in the context of the collection as well as photos captured in the same scene. (a) Extract from a photo collection and visualization of clustering. (b) Photos captured in the same scene and close-ups of image details. (c) Top: image labeling obtained from independent assessment. Bottom: labels assigned after our context-aware adaptation of the independent score.
HAL (Le Centre pour la Communication Scientifique Directe), Oct 4, 2022
Gaze behavior of virtual characters in video games and virtual reality experiences is a key facto... more Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.
The hybrid point-source/wave-field method is a newly proposed approach for Computer-Generated Hol... more The hybrid point-source/wave-field method is a newly proposed approach for Computer-Generated Hologram (CGH) calculation, based on the slicing of the scene into several depth layers parallel to the hologram plane. The complex wave scattered by each depth layer is then computed using either a wave-field or a point-source approach according to a threshold criterion on the number of points within the layer. Finally, the complex waves scattered by all the depth layers are summed up in order to obtain the final CGH. Although outperforming both point-source and wave-field methods without producing any visible artifact, this approach has not yet been used for animated holograms, and the possible exploitation of temporal redundancies has not been studied. In this paper, we propose a fast computation of video holograms by taking into account those redundancies. Our algorithm consists of three steps. First, intensity and depth data of the current 3D video frame are extracted and compared with those of the previous frame in order to remove temporally redundant data. Then the CGH pattern for this compressed frame is generated using the hybrid point-source/wave-field approach. The resulting CGH pattern is finally transmitted to the video output and stored in the previous frame buffer. Experimental results reveal that our proposed method is able to produce video holograms at interactive rates without producing any visible artifact.
We propose a new Tone Mapping Operator dedicated to the visualization of 360 • High Dynamic Range... more We propose a new Tone Mapping Operator dedicated to the visualization of 360 • High Dynamic Range images on Head-Mounted Displays. Previous work around this topic has shown that the existing Tone Mapping Operators for classic 2D images are not adapted to 360 • High Dynamic Range images. Consequently, several dedicated operators have been proposed. Instead of operating on the entire 360 • image, they only consider the part of the image currently viewed by the user. Tone mapping a part of the 360 • image is less challenging as it does not preserve globally the dynamic range of the luminance of the scene. To cope with this problem, we propose a novel Tone Mapping Operator which takes advantage of 1) a view-dependant tone mapping that enhances the contrast, and 2) a Tone Mapping Operator applied to the entire 360 • image that preserves the global coherency. Furthermore, the proposed Tone Mapping Operator is adapted to the human eye perception of the luminance on Head-Mounted Displays. We present two subjective studies to model the lightness perception on such Head-Mounted Displays.
ACM Transactions on Applied Perception, Apr 30, 2019
Recent progress in digital photography and storage availability has drastically changed our appro... more Recent progress in digital photography and storage availability has drastically changed our approach to photo creation. While in the era of film cameras, careful forethought would usually precede the capture of a photo; nowadays, a large number of pictures can be taken with little effort. One of the consequences is the creation of numerous photos depicting the same moment in slightly different ways, which makes the process of organizing photos laborious for the photographer. Nevertheless, photo collection organization is important both for exploring photo albums and for simplifying the ultimate task of selecting the best photos. In this work, we conduct a user study to explore how users tend to organize or cluster similar photos in albums, to what extent different users agree in their clustering decisions, and to investigate how the clustering-defined photo context affects the subsequent photo-selection process. We also propose an automatic hierarchical clustering solution for modeling user clustering decisions. To demonstrate the usefulness of our approach, we apply it to the task of automatic photo evaluation within photo albums and propose a clustering-based context adaptation.
A hybrid approach for fast occlusion processing in computer generated hologram calculation is stu... more A hybrid approach for fast occlusion processing in computer generated hologram calculation is studied in this paper. The proposed method is based on the combination of two commonly used approaches, which complement one another: the point-source and wave-field approaches. By using these two approaches together, the proposed method thus takes advantages from both of them. In this method, the 3D scene is first sliced into several depth layers parallel to the hologram plane. Light scattered by the scene is then propagated and shielded from one layer to another using either a point-source or a wave-field approach according to a threshold criterion on the number of points within the layer. Finally, the hologram is obtained by computing the propagation of light from the nearest layer to the hologram plane. Experimental results reveal that the proposed method does not produce any visible artifact and outperforms both the point-source and wave-field approaches.
The selection of the best photos in personal albums is a task that is often faced by photographer... more The selection of the best photos in personal albums is a task that is often faced by photographers. This task can become laborious when the photo collection is large and it contains multiple similar photos. Recent advances on image aesthetics and photo importance evaluation has led to the creation of different metrics for automatically assessing a given image. However, these metrics are intended for the independent assessment of an image, without considering the possible context implicitly present within photo albums. In this work, we perform a user study for assessing how users select photos when provided with a complete photo album-a task that better reflects how users may review their personal photos and collections. Using the data provided by our study, we evaluate how existing state-of-the-art photo assessment methods perform relative to user selection, focusing in particular on deep learning based approaches. Finally, we explore a recent framework for adapting independent image scores to collections and evaluate in which scenarios such an adaptation can prove beneficial.
HAL (Le Centre pour la Communication Scientifique Directe), Oct 7, 2015
We propose a fast Computer-Generated Hologram (CGH) computation method with occlusion effect base... more We propose a fast Computer-Generated Hologram (CGH) computation method with occlusion effect based on a hybrid point-source/wave-field approach. Whereas previously proposed methods tried to reduce the computational complexity of the point-source or the wave-field approaches independently, the proposed method uses the two approaches together and therefore takes advantages from both of them. Our algorithm consists of three steps. First, the 3D scene is sliced into several depth layers parallel to the hologram plane. Then, light scattered by the scene is propagated and shielded from one layer to another, starting from the farthest layer. For each layer, light propagation and light shielding are performed using either a point-source or a wave-field approach according to a threshold criterion on the number of points within the layer. Finally, we compute light propagation from the nearest layer to the hologram plane in order to obtain the final CGH. Experimental results reveal that this combination of approaches does not produce any visible artifact and outperforms both the point-source and wave-field approaches.
We propose a Tone Mapping Operator, denoted HMD-TMO, dedicated to the visualization of 360 • High... more We propose a Tone Mapping Operator, denoted HMD-TMO, dedicated to the visualization of 360 • High Dynamic Range images on Head Mounted Displays. The few existing studies about this topic have shown that the existing Tone Mapping Operators for classic 2D images are not adapted to 360 • High Dynamic Range images. Consequently, several dedicated operators have been proposed. Instead of operating on the entire 360 • image, they only consider the part of the image currently viewed by the user. Tone mapping a part of the 360 • image is less challenging as it does not preserve the global luminance dynamic of the scene. To cope with this problem, we propose a novel tone mapping operator which takes advantage of both a view-dependant tone mapping that enhances the contrast, and a Tone Mapping Operator applied to the entire 360 • image that preserves global coherency. Furthermore, we present a subjective study to model lightness perception in a Head Mounted Display.
In this paper we propose a method for recovering the shape (geometry) and the diffuse reflectance... more In this paper we propose a method for recovering the shape (geometry) and the diffuse reflectance from an image (or video) using a hybrid setup consisting of a depth sensor (Kinect), a consumer camera and a partially controlled illumination (using a flash). The objective is to show how combining RGB-D acquisition with a sequential illumination is useful for shape and reflectance recovery. A pair of two images are captured: one non flashed (image under ambient illumination) and a flashed one. A pure flash image is computed by subtracting the non flashed image from the flashed image. We propose an novel and near real-time algorithm, based on a local illumination model of our flash and the pure flash image, to enhance geometry (from the noisy depth map) and recover reflectance information.
HAL (Le Centre pour la Communication Scientifique Directe), Nov 25, 2014
Cet article présente un état de l'art de la génération numérique d'hologrammes. Les méthodes prés... more Cet article présente un état de l'art de la génération numérique d'hologrammes. Les méthodes présentées permettent de générer l'hologramme d'une scène synthétique ou réelle sans passer par le processus physique réel d'interférence entre deux ondes lumineuses. Les principales limitations liées à l'holographie conventionnelle : (1) la nécessité d'utiliser une source laser cohérente et (2) l'obligation d'avoir un système optique extrêmement stable peuvent ainsi être évitées. This paper reviews Computer Generated Holography techniques. Using these methods, it is possible to acquire holograms of synthetic or existing scenes without physical interference between light waves. Most limitations characterizing conventional holography, namely the need for a powerful, highly coherent laser and extreme stability of the optical system are thus avoided.
IEEE Transactions on Visualization and Computer Graphics
Gaze behavior of virtual characters in video games and virtual reality experiences is a key facto... more Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.
Proceedings of the symposium on Computational Aesthetics, 2017
Figure 1: Our context-aware framework performs the assessment and labeling of images in photo col... more Figure 1: Our context-aware framework performs the assessment and labeling of images in photo collections by considering image quality in the context of the collection as well as photos captured in the same scene. (a) Extract from a photo collection and visualization of clustering. (b) Photos captured in the same scene and close-ups of image details. (c) Top: image labeling obtained from independent assessment. Bottom: labels assigned after our context-aware adaptation of the independent score.
Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, 2018
The selection of the best photos in personal albums is a task that is often faced by photographer... more The selection of the best photos in personal albums is a task that is often faced by photographers. This task can become laborious when the photo collection is large and it contains multiple similar photos. Recent advances on image aesthetics and photo importance evaluation has led to the creation of different metrics for automatically assessing a given image. However, these metrics are intended for the independent assessment of an image, without considering the possible context implicitly present within photo albums. In this work, we perform a user study for assessing how users select photos when provided with a complete photo album-a task that better reflects how users may review their personal photos and collections. Using the data provided by our study, we evaluate how existing state-of-the-art photo assessment methods perform relative to user selection, focusing in particular on deep learning based approaches. Finally, we explore a recent framework for adapting independent image scores to collections and evaluate in which scenarios such an adaptation can prove beneficial.
We propose a new approach for real-time Tone Mapping Operator dedicated to High Dynamic Range ren... more We propose a new approach for real-time Tone Mapping Operator dedicated to High Dynamic Range rendering of interactive 3D scenes. The proposed method considers the whole scene lighting in order to preserve the global coherency. This is the major contribution of our method. Indeed, most of existing Tone Mapping Operators only consider the image rendered by the camera at the current frame and simulate the Human Visual System accommodation to bright and dark luminance. Consequently, after an adaptation time, the lighting design of the 3D scene is lost. For example, two rooms with a high contrast (one dark and one bright) can be perceived with the same luminance level after adaptation. To cope with this coherency issue, we adapt an existing Tone Mapping Operator that combines (1) a global Tone Mapping Operator which takes into account the High Dynamic Range of the whole scene and (2) a viewport frame based Tone Mapping Operator that enhances the image contrast. Thus, we preserve the global lighting coherency and enhance the contrast for each rendered image. Furthermore, we present a subjective evaluation that shows that our method provides a better user experience than previous methods in the case of visualization on Head-Mounted Display.
Figure 1: The 3-point-lighting technique (on the left) is a classical cinematic rule to place lig... more Figure 1: The 3-point-lighting technique (on the left) is a classical cinematic rule to place lights around a subject. Playing with the lights position, size, and flux considerably impacts the aesthetics and atmosphere of a scene (in the middle). We propose a technique that automatically computes the parameters of a 3-point-lighting setup made of area lights, given the reference image in the middle, to generate the image on the right in a virtual environment.
Abstract To display high dynamic range videos on a low dynamic range display, a tone mapping oper... more Abstract To display high dynamic range videos on a low dynamic range display, a tone mapping operation is needed. Tone mapping independently each frame of a video sequence leads to temporal artifacts that impair the visual quality of the resulting tone-mapped video. These temporal artifacts are classified into six categories: global flickering, local flickering, temporal noise, temporal brightness incoherency, temporal object incoherency, and hue coherency. We explain in detail the situations in which these artifacts may occur. We review existing video tone mapping operators (techniques that take into account more than a single frame) and show which artifacts are handled by these operators.
Uploads
Papers by Remi Cozot