Tobias Weyand

Hi there! I’m a Computer Vision researcher at Google DeepMind working on video understanding, with a focus on learning long video representations. Before that, I dabbled in landmark recognition, image localization, large-scale image retrieval and clustering. Prior to joining Google, I did my PhD at the RWTH Aachen Vision Lab with Bastian Leibe. I earned my Dipoma (MS) in Computer Science from RWTH Aachen University.

In a past life, I worked on some Nintendo DS homebrew projects that can be found on my Github page.

selected publications

Neptune: The Long Orbit to Benchmarking Long Video Understanding

Arsha Nagrani , Mingda Zhang , Ramin Mehran , and 10 more authors

2024

arXiv Bib Blog Website

@misc{nagrani2024neptune,
  title = {Neptune: The Long Orbit to Benchmarking Long Video Understanding},
  author = {Nagrani, Arsha and Zhang, Mingda and Mehran, Ramin and Hornung, Rachel and Gundavarapu, Nitesh Bharadwaj and Jha, Nilpa and Myers, Austin and Zhou, Xingyi and Gong, Boqing and Schmid, Cordelia and Sirotenko, Mikhail and Zhu, Yukun and Weyand, Tobias},
  year = {2024},
}

Extending Video Masked Autoencoders to 128 frames

Nitesh Bharadwaj Gundavarapu , Luke Friedman , Raghav Goyal , and 8 more authors

In Conference on Neural Information Processing Systems , 2024

arXiv Bib Blog

@inproceedings{gundavarapu2024lvmae,
  title = {Extending Video Masked Autoencoders to 128 frames},
  author = {Gundavarapu, Nitesh Bharadwaj and Friedman, Luke and Goyal, Raghav and Hegde, Chaitra and Agustsson, Eirikur and Waghmare, Sagar M. and Sirotenko, Mikhail and Yang, Ming-Hsuan and Weyand, Tobias and Gong, Boqing and Sigal, Leonid},
  year = {2024},
  booktitle = {Conference on Neural Information Processing Systems},
}

VideoPrism: A Foundational Visual Encoder for Video Understanding

Long Zhao , Nitesh Bharadwaj Gundavarapu , Liangzhe Yuan , and 16 more authors

In International Conference on Machine Learning , 2024

arXiv Bib

@inproceedings{zhao2024videoprism,
  title = {VideoPrism: A Foundational Visual Encoder for Video Understanding},
  author = {Zhao, Long and Gundavarapu, Nitesh Bharadwaj and Yuan, Liangzhe and Zhou, Hao and Yan, Shen and Sun, Jennifer J. and Friedman, Luke and Qian, Rui and Weyand, Tobias and Zhao, Yue and Hornung, Rachel and Schroff, Florian and Yang, Ming and Ross, David A. and Wang, Huisheng and Adam, Hartwig and Sirotenko, Mikhail and Liu, Ting and Gong, Boqing},
  booktitle = {International Conference on Machine Learning},
  year = {2024},
}

VideoGLUE: Video General Understanding Evaluation of Foundation Models

Liangzhe Yuan , Nitesh Bharadwaj Gundavarapu , Long Zhao , and 14 more authors

Transactions on Machine Learning Research, 2024

arXiv Bib

@article{yuan2024videoglue,
  title = {Video{GLUE}: Video General Understanding Evaluation of Foundation Models},
  author = {Yuan, Liangzhe and Gundavarapu, Nitesh Bharadwaj and Zhao, Long and Zhou, Hao and Cui, Yin and Jiang, Lu and Yang, Xuan and Jia, Menglin and Weyand, Tobias and Friedman, Luke and Sirotenko, Mikhail and Wang, Huisheng and Schroff, Florian and Adam, Hartwig and Yang, Ming-Hsuan and Liu, Ting and Gong, Boqing},
  journal = {Transactions on Machine Learning Research},
  issn = {2835-8856},
  year = {2024},
}

Google Landmarks Dataset v2 - A Large-Scale Benchmark for Instance-Level Recognition and Retrieval

Tobias Weyand, André Araujo , Bingyi Cao , and 1 more author

In IEEE Conference on Computer Vision and Pattern Recognition , 2020

arXiv Bib Website

@inproceedings{weyand2020gldv2,
  title = {Google Landmarks Dataset v2 - A Large-Scale Benchmark for Instance-Level Recognition and Retrieval},
  author = {Weyand, Tobias and Araujo, André and Cao, Bingyi and Sim, Jack},
  year = {2020},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
}

PlaNet - Photo Geolocation with Convolutional Neural Networks

Tobias Weyand, Ilya Kostrikov , and James Philbin

In European Conference on Computer Vision , 2016

arXiv Bib Website

@inproceedings{weyand2016planet,
  title = {PlaNet - Photo Geolocation with Convolutional Neural Networks},
  author = {Weyand, Tobias and Kostrikov, Ilya and Philbin, James},
  year = {2016},
  booktitle = {European Conference on Computer Vision},
}

Discovering Details and Scene Structure with Hierarchical Iconoid Shift

Tobias Weyand, and Bastian Leibe

In International Conference on Computer Vision , 2013

Bib PDF

@inproceedings{Weyand13ICCV,
  author = {Weyand, Tobias and Leibe, Bastian},
  title = {{Discovering Details and Scene Structure with Hierarchical Iconoid Shift}},
  booktitle = {International Conference on Computer Vision},
  year = {2013},
}

Image Retrieval for Image-Based Localization Revisited

Torsten Sattler , Tobias Weyand, Bastian Leibe , and 1 more author

In British Machine Vision Conference , 2012

Bib PDF

@inproceedings{Sattler12BMVC,
  author = {Sattler, Torsten and Weyand, Tobias and Leibe, Bastian and Kobbelt, Leif},
  title = {{Image Retrieval for Image-Based Localization Revisited}},
  booktitle = {British Machine Vision Conference},
  year = {2012},
}

Discovering Favorite Views of Popular Places with Iconoid Shift

Tobias Weyand, and Bastian Leibe

In International Conference on Computer Vision , 2011

Bib PDF

@inproceedings{Weyand11ICCV,
  author = {Weyand, Tobias and Leibe, Bastian},
  title = {{Discovering Favorite Views of Popular Places with Iconoid Shift}},
  booktitle = {International Conference on Computer Vision},
  year = {2011},
}