Wilfredo Torres Calderon

I am a Post Doctoral Researcher at the University of Illinois at Urbana-Champaign, where I conduct research in computer vision, multimodal vision-language models (VLMs), and video understanding. I am advised by Derek Hoiem.

I completed my Ph.D. in Civil Engineering at UIUC, where I was advised by Mani Golparvar-Fard. Before that, I completed an MSc in Civil Engineering at UIUC, and simultaneously with my Ph.D. studies, I pursued an MCS in Computer Science at UIUC. I earned my bachelor's degree at the Pontificia Universidad Católica del Perú.

Email  /  CV  /  Google Scholar

My research spans computer vision, multimodal learning, and continual learning, with emphasis on systems that operate over long temporal horizons. I have contributed to the development of next-generation vision-language models through:

• Designing models for long-term spatiotemporal reasoning and open-vocabulary video understanding.
• Developing object- and part-level representations for explainable visual reasoning in dynamic scenes.
• Building continual learning frameworks that allow models to accumulate knowledge over time without retraining from scratch.
• Advancing multimodal architectures capable of integrating vision, language, and long-video context.
• Working on DARPA-sponsored research targeting robust perception and adaptive learning in real-world environments.

My interdisciplinary background in Civil Engineering and Computer Science informs my approach to modeling physical environments, structured geometry, and dynamic processes, enabling the creation of robust and interpretable vision systems.

Region representations used to be popular in the pre-deep learning era. What happens when we create region representations with recently released foundation models? We show that our simple method achieves impressive performance on existing tasks such as semantic segmentation as well as new one.

A vision-based activity analysis method that leverages synthetically generated hauling operations using 3D simulations.

A vision-based activity analysis method that leverages 2D pose estimation outputs that are used in many state-of-the-art construction worker ergonomics analysis methods.

A 2D human pose annotation tool adapted from CVAT that can also annotate per-frame activity labels.

An NLP model for automated mapping of raw construction activities to 3D BIM elements.

Credit to Jon Barron .