Mritunjoy Halder

Large 3D shape repositories are rapidly expanding, driven by advances in generative modeling, making efficient shape retrieval increasingly important for authoring tools. While text queries capture high-level semantics, they often fail to convey precise geometric details. 3D sketches provide a more expressive means of representing shape geometry, and recent AR/VR developments have made sketch-based retrieval practical. However, existing 3D sketch datasets face three major limitations: (1) reliance on quad meshes or voxel hulls, which often fail on complex or non-manifold shapes; (2) use of fixed-size point clouds that discard stroke connectivity and limit geometric fidelity; and (3) dependence on expensive curve-based or multi-view rendering pipelines, which hinder large-scale data generation. Limited point cloud representations also fail to capture sketch connectivity and topology when used to train retrieval models. To address these challenges, we propose Sketch3R, a scalable framework that converts arbitrary 3D meshes into human-like VR sketches using a graph-based representation that preserves stroke connectivity and adapts to sketch complexity. Leveraging this representation, Sketch3R employs a lightweight graph-attention Siamese network for efficient and accurate sketch-to-shape retrieval. Experiments demonstrate that our method outperforms prior approaches in both accuracy and speed, while robustly handling 3D shapes across diverse topologies.

This paper introduces a novel mixed-reality (MR) 3D authoring platform designed to enable real-time, interactive composition of 3D scenes in augmented and mixed reality environments. The platform proposed and integrates two key innovations: a three-tier 3D asset generation workflow and a 3D sketch-to-shape retrieval algorithm. The three-tier architecture addresses the challenge of time-consuming 3D asset creation by allowing users to engage with the generation process at intermediate stages, enabling rapid validation and refinement of user inputs. This approach not only enhances interactivity but also accelerates the realization of user imagined assets. Complementing this is our 3D sketch-to-shape retrieval system, which leverages an intuitive, robust sketching module to query a library of generated or artist-created 3D assets. This mechanism facilitates precise retrieval of objects matching both the category and rough shape of a user drawn 3D sketch of the assets. The platform supports multi-modal, intuitive user inputs such as speech, text, and freehand 3D sketches, making it accessible to both novice and expert users for creating, retrieving, modifying, and composing 3D assets and textures. By combining generative models with intuitive interaction, the platform serves as a flexible testbed for 3D asset development and a tool to democratize 3D scene authoring. We show effectiveness of GeMR on various use-cases, including scenario composition for human-robot collaboration. We conducted a user study to evaluate whether our system effectively democratizes the 3D scene composition. We compared the performance of 3D modeling experts and novice users on a standardized scene creation task. Both novice and expert users reported equally good usability ratings and low perceived workload. Our findings highlight that an integration of generative models and multimodal intuitive inputs represents a significant leap in making the 3D scene authoring more accessible.

Most of the current diffusion-based surface normal estimators were trained at low resolutions (e.g., mathematical equation), significantly below the resolution of modern cameras. Applying such models to high-resolution images causes two issues: (1) downsampling removes high-frequency geometric details, and (2) direct high-resolution inference loses global context. To address these challenges, we propose UltraNormal, which augments existing pretrained diffusion-based surface normal estimators by using overlapping sliding patches with test-time guidance to inject global context. UltraNormal initializes each patch with the latent of the corresponding low-resolution estimate enhanced with high-frequency details from the local estimate and resumes the reverse diffusion process from an intermediate time step. This approach maintains geometric accuracy of the global prediction while adding high-frequency details without retraining the diffusion model. To the best of our knowledge, this is the first test-time method to scale low-resolution diffusion-based surface normal estimators to 4K resolution without any retraining. We build UltraNormal upon three different base models and evaluate on both real-world data and a 4K photorealistic synthetic dataset. We perform extensive ablations and demonstrate improved accuracy, sharpness, and surface reconstruction quality compared to the base model.

Human-robot collaboration is essential in AI-powered Industry 5.0 and personal spaces, aiming to boost human creativity and efficiency. Traditional methods involve creating virtual scenarios in VR to simulate interactions, but they often rely on pre-defined 3D assets or procedural generation, which are time-consuming, expensive, and limited by restricted design options and the need for 3D expertise. Many systems are also desktop-based and lack intuitive interfaces, limiting interactive scenario creation and in-place testing. This paper presents an integrated XR-based system that combines GenAI based 3D scenario generation and human-robot collaboration in real-time. It allows users to create and modify virtual environments on-the-fly using voice and gesture inputs, and test collaborations directly in mixed reality, accelerating decision-making. The system, powered by Generative AI and XR, enables even non-experts to generate 3D assets, assemble them, configure robots, and perform collaborative tasks. It features an intuitive interface and multiple configuration options. Through experiments and ablations, the system’s effectiveness is validated, showing its potential for interactive 3D scenario creation and in-place collaboration in XR, with applications in robotics, gaming and other fields.

In recent years, one of the main contributors to traffic fatalities and poor vision in poor weather conditions has been rising. In order to lower accident rates, improving visibility is essential for driver assistance systems, image acquisition, and surveillance systems. Thus, as a result, this research introduces a novel Deep Neural Network (DNN) based on a scattering model for defogging an image that has been affected with fog. The proposed model consists of two components: the fog dilution model and the diluted fog removal model. In addition, we have incorporated a deep learning-based transmission estimation module. The fog dilution model is designed to mitigate the presence of fog, while the diluted fog removal model aims to completely eliminate the remaining fog. To enhance the accuracy of transmission estimation, we employ a green channel prior-based approach. This approach effectively reduces the distortion caused in the sky in the resulting defogged image. By combining these components, our model offers a comprehensive solution for fog removal, with improved image quality and reduced sky artifacts. In addition to addressing the shortcomings of current vision enhancement techniques as mentioned in the paper, the proposed method also satisfies human perceptual needs. Experimental results have demonstrated that the proposed model delivers superior visibility enhancement results for both non-reference and full-reference metrics following a qualitative comparison with eight state-of-the-art defogging techniques. The experimental results have proven that the proposed method achieves superior performance in terms of qualitative evaluation on non-reference metric (i.e., in terms of e = 0.50, = 0.15, r = 1.49) and reference metric (i.e. in terms of MSE= 397.16, PSNR = 23.38, NCC = 0.99, MD = 24.39, NAE = 0.13) compared with eight state-of-the-art dehazing methods. Furthermore, based on the average computational time achieved by the proposed method (0.17 s using HSTS dataset), it can be highly suitable for real-time applications. The suggested approach could therefore be employed as a viable route for improving vision in surveillance and acquisition systems while lowering the risk to users’ safety.