Mritunjoy Halder

With the increasing demand for 3D content in AR/VR applications, 3D shape libraries are expanding rapidly. As a result, the ability to retrieve 3D shapes from these libraries has become an essential component. It is well established that text-based queries are insufficient for accurately describing geometric shapes, and 2D sketches lack the necessary detail. In contrast, 3D sketches are more effective at conveying geometric properties. In this paper, we introduce a new learning-based approach for retrieving 3D shapes from 3D VR sketches. Since there is a lack of large-scale datasets containing paired 3D VR sketches and 3D shapes, our second contribution is a novel data generation technique that automatically converts 3D shapes into human-like 3D VR sketches. We conduct a comprehensive evaluation of our data generation and retrieval methods against state-of-the-art approaches, demonstrating that our method is both efficient and achieves superior retrieval accuracy.

Human-robot collaboration is essential in AI-powered Industry 5.0 and personal spaces, aiming to boost human creativity and efficiency. Traditional methods involve creating virtual scenarios in VR to simulate interactions, but they often rely on pre-defined 3D assets or procedural generation, which are time-consuming, expensive, and limited by restricted design options and the need for 3D expertise. Many systems are also desktop-based and lack intuitive interfaces, limiting interactive scenario creation and in-place testing. This paper presents an integrated XR-based system that combines GenAI based 3D scenario generation and human-robot collaboration in real-time. It allows users to create and modify virtual environments on-the-fly using voice and gesture inputs, and test collaborations directly in mixed reality, accelerating decision-making. The system, powered by Generative AI and XR, enables even non-experts to generate 3D assets, assemble them, configure robots, and perform collaborative tasks. It features an intuitive interface and multiple configuration options. Through experiments and ablations, the system’s effectiveness is validated, showing its potential for interactive 3D scenario creation and in-place collaboration in XR, with applications in robotics, gaming and other fields.

In recent years, one of the main contributors to traffic fatalities and poor vision in poor weather conditions has been rising. In order to lower accident rates, improving visibility is essential for driver assistance systems, image acquisition, and surveillance systems. Thus, as a result, this research introduces a novel Deep Neural Network (DNN) based on a scattering model for defogging an image that has been affected with fog. The proposed model consists of two components: the fog dilution model and the diluted fog removal model. In addition, we have incorporated a deep learning-based transmission estimation module. The fog dilution model is designed to mitigate the presence of fog, while the diluted fog removal model aims to completely eliminate the remaining fog. To enhance the accuracy of transmission estimation, we employ a green channel prior-based approach. This approach effectively reduces the distortion caused in the sky in the resulting defogged image. By combining these components, our model offers a comprehensive solution for fog removal, with improved image quality and reduced sky artifacts. In addition to addressing the shortcomings of current vision enhancement techniques as mentioned in the paper, the proposed method also satisfies human perceptual needs. Experimental results have demonstrated that the proposed model delivers superior visibility enhancement results for both non-reference and full-reference metrics following a qualitative comparison with eight state-of-the-art defogging techniques. The experimental results have proven that the proposed method achieves superior performance in terms of qualitative evaluation on non-reference metric (i.e., in terms of e = 0.50, = 0.15, r = 1.49) and reference metric (i.e. in terms of MSE= 397.16, PSNR = 23.38, NCC = 0.99, MD = 24.39, NAE = 0.13) compared with eight state-of-the-art dehazing methods. Furthermore, based on the average computational time achieved by the proposed method (0.17 s using HSTS dataset), it can be highly suitable for real-time applications. The suggested approach could therefore be employed as a viable route for improving vision in surveillance and acquisition systems while lowering the risk to users’ safety.

Can a surveillance robot autonomously detect anomalous activity from its ego view camera perception? This is a challenging task as it requires identifying what is normal and what is an abnormal pattern - given the variations of possible anomalies and abnormalities. This paper presents an architecture and method based on a spatio-temporal convolution neural network to detect and classify anomalies. This work is inspired by the ‘Konio-Magno-Parvocellular’ cells of the human brain, which is claimed to aid humans in organizing changes in perceived scenes. The model is trained and tested on a benchmark video dataset of human activity. We have obtained 91% testing accuracy on this dataset. Experiments in simulation as well as deployment on a real robot shows that the proposed methodology can identify anomalous activities effectively. We have also listed down the observations from practical deployment of the model.