Dinesh Babu Jayagopi

Associate Professor

Email: jdinesh@iiitb.ac.in

Education

  • Ph.D. (École Polytechnique Fédérale de Lausanne)

Dr. Dinesh Babu Jayagopi is an Assistant Professor at IIIT Bangalore since Dec 2013, where he heads the Multimodal Perception Lab.  His research interests are in Audio-Visual Signal Processing, Machine Learning, and Social Computing. He obtained his doctorate from Ecole Polytechnic Federale Lausanne (EPFL), Switzerland, beginning of 2011. He received the Outstanding paper award in the International Conference on Multimodal Interaction (ICMI), 2012, Idiap Best PhD student research award for the year 2009. More recently, his PhD student's work was nominated for Best student paper in ICMI 2016 Tokyo, Japan. Another work received Best student paper award at MedPRAI 2018. He also received  Department of Science and Technology (DST) Young Scientist Start up Grant in 2016. In the past, he has successfully collaborated and executed sponsored research projects with CAIR, DRDO (Defence Research and Development Organization) and NI Systems. He was a visiting professor at University of Lausanne in summer 2019. 

Research Interests

  • Audio Visual Signal, Processing, Applied Machine Learning, Machine Perception, Applied Signal Processing, Social Computing, Human Robot Interaction, Multimedia Analysis, Behavior Analysis

Selected Publications

Book Chapter :

  • O. Aran, D. Gatica Perez, and D. Jayagopi . Understanding small groups , Social Signal Processing, Cambridge University Press 2017. Editors J. Burgoon A. Vinciarelli, M. Pantic and N. Magnenat Thalmann. [Collaboration with EPFL]

Journal

  • Manjunath K E, D. Jayagopi, K S Rao, Ramasubramanian V Development and Analysis of Multilingual Phone Recognition Systems using Indian Languages, accepted for publication in Springer International Journal of Speech Technology (Impact Factor: 0.9).

  • S. Rasipuram and D. Jayagopi. Automatic assessment of communication skill in interview-based interactions,  accepted  for publication in Springer Multimedia Tools and Applications 2018 (Impact Factor:  1.5).

  • D. Sanches-Cortes, O. Aran, D. Jayagopi, M. Schmid-Mast, and D. Gatica-Perez. Emergent leaders through looking and speaking: from audio-visual data to multimodal recognition, accepted for publication in Springer Journal on Multimodal User Interfaces, 2012.
     
  • D. Jayagopi, T. Kim, Alex (Sandy) Pentland, and D. Gatica-Perez. Privacy-sensitive recognition of group conversational context with sociometers, Springer Multimedia Systems, 2012, Volume 18, Number 1, Pages 3-14. 
     
  • D. Jayagopi, and D. Gatica-Perez. Mining group nonverbal conversational patterns using probabilistic topic models, IEEE Transactions on Multimedia, 2010, 12(8), 790-802.
     
  • D. Jayagopi, H. Hung, C. Yeo, and D. Gatica-Perez. Modeling dominance in group conversations from nonverbal activity cues, IEEE Transactions on Audio, Speech and Language Processing, 2009,17(3), 501-513.

Conference

  • S. Rasipuram and D. Jayagopi. A Comprehensive Evaluation of Audio-Visual Behavior in Various Modes of Interviews in the Wild, accepted for oral presentation at the 12th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA), June 2019, Rhodes, Greece.
  • C. Thomas, D. Gatica-Perez and D. Jayagopi. BookTubing Across Regions: Examining Differences based on Nonverbal and Verbal Cues, accepted for publication in ACM International Conference on Interactive Experiences for Television and Online Video (TVX), June 2019, Manchester, UK. Collaboration with IDIAP, EPFL
  • S. Raju, N. Duvvuru and D. Jayagopi. Improving Adversarial Images Using Activation Maps, accepted for publication in 8th Joint International Information Technology and Artificial Intelligence Conference (IEEE ITAIC 2019), May 2019, Chongqing,China.
  • A. Sharma, D. Jayagopi. Handwritten Essay Grading on Mobiles using MDLSTM Model and Word Embeddings, published in the 11th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), Hyderabad, India, Dec 2018.
  • C. Nagraj, H. Jain, D. Jayagopi. Rash Driving Detection using a Frontal View Camera in Cars, published in the 11th Indian Conference on Computer Vision, Graphics and Image Processing (ICVGIP), Hyderabad, India, Dec 2018. Collaboration with Samsung R&D
  • D. Gatica-Perez, D. Cortez, T. Minh-Do, K. Otsuka, D. Jayagopi. Vlogging Over Time: Longitudinal Impressions and Behavior in YouTube, accepted in the 17th International Conference on Mobile and Ubiquitous Multimedia (MuM), Cairo, Egypt, Nov 2018.
  • N. Nair, C. Thomas and D. Jayagopi. Human Activity Recognition Using Temporal Convolutional Network., published in the 5th International Workshop on Sensor-based Activity Recognition and Interaction (iWOAR) 2018, Berlin Sep 2018.
  • K E Manjunath, K S Rao, D. Jayagopi, V. Ramasubramanian Indian languages ASR: A multilingual phone recognition framework with IPA based common phone-set, predicted articulatory features and feature fusion., published in the Proceedings of Interspeech, Sep 2018, Hyderabad.
  • A. Gupta and D. Jayagopi. Unsupervised Speaker Cue Usage Detection in Public Speaking Videos, accepted in the 1st International Workshop on Vision for Interaction and Behaviour undErstanding (VIBE 2018), BMVC, New Castle, Sep 2018.
  • A. Sharma and D. Jayagopi. Automated Grading of Handwritten Essays , published in  16th International Conference on Frontiers in Handwriting Recognition (ICFHR-2018), to be held August 5-8, 2018 in Niagara Falls, USA.
  • S. Praneeth, U. Agrawal and D. Jayagopi. Human Weapon-Activity Recognition in Surveillance Videos Using Structural-RNN, published in the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI’2018), to be held in Rabat, Morocco, on March 27-28, 2018. Best student paper award!!
  • A. Surendranath and D. Jayagopi. Curriculum Learning for Depth Estimation With Deep Convolutional Neural Networks, published in the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (MedPRAI’2018), to be held in Rabat, Morocco, on March 27-28, 2018.
  • P. Sudheendra and D. Jayagopi. Genre Linked Automated Assessment and feedback of Photographs based on Visual Aesthetics, published in the International Conference on Image Processing Tools and Applications (IPTA), Montreal, Nov 2017.
  • P. Rao, S. Rasipuram, R. Das, D. Jayagopi. Automatic Assessment of Communication Skill in Non-Conventional Interview Settings: A Comparative Study, published in International Conference on Multimodal Interaction(ICMI), Glasgow, Scotland, Nov 2017 Pooja R gets Travel Grant!!.
  • C.Thomas, D. Jayagopi. Predicting Student Engagement in Classrooms Using Facial Behavioral Cues, published in ICMI MIE Workshop Glasgow, Scotland, Nov 2017.
  • S.K. Nambiar, R. Das, S. Rasipuram, D. Jayagopi. Automatic Generation of Actionable Feedback Towards Improving Social Competency in Job Interviews., published in ICMI MIE Workshop Glasgow, Scotland, Nov 2017.
  • S. Rasipuram, R. Das, P. Rao, D. Jayagopi. Online Peer-to-peer Discussions: A Platform for Automatic Assessment of Communication Skill, published in Context Based Affect Recognition Workshop, ACII, San Antonio, Texas, USA, Oct 2017.
  • K. E. Manjunath, K.S. Rao  and D. Jayagopi. Development of Multilingual Phone Recognition System for Indian Languages IEEE SPICES Kochi, Aug 2017.
  • M.K. Prabhu and D. Jayagopi. MERS – A Multimodal Real Time Emotion Recognition System published in the Second International Workshop on Pattern Recognition (IWPR) Singapore, May 2017.
  • P. Dhananjay, D. Varshneya and D. Jayagopi. Restaurant Attribute classification using Deep Learning published in the Proceedings of INDICON, Bangalore, Dec 2016.
  • S. Rasipuram, P. Rao, and D. Jayagopi. Automatic Prediction of Fluency in Interface-based Interviews published in the Proceedings of INDICON, Bangalore, Dec 2016.
  • M. Madhyastha and D. Jayagopi. A Low Cost Personalised Robot Language Tutor with Perceptual and Interaction Capabilities published in the Proceedings of INDICON, Bangalore, Dec 2016.
  • S. Rasipuram, P. Rao, D. Jayagopi. Asynchronous Video Interviews vs Face-to-Face Interviews For Communication Skill Measurement: A Systematic Study, accepted in International Conference  on Multimodal Interaction(ICMI),  Tokyo, Japan, Nov 2016. Best Student Paper Nomination.
  • G. GopanK, N. Sinha, D. Jayagopi. Statistical Feature Analysis for EEG Baseline Classification : Eyes Open vs Eyes Closed, IEEE TENCON Nov 2016, Singapore.
  • P. Venkatesh, D. Jayagopi. Automatic Expression Recognition and Expertise Prediction in Bharatnatyam, published in the International Conference on Advances in Computing, Communications and Informatics (ICACCI), Sep 2016, LNMIIT Jaipur.
  • P. Venkatesh, D. Jayagopi. Automatic Bharatnatyam Dance Posture Recognition and Expertise Prediction using Depth Cameras, published in the SAI Intelligent Systems Conference 2016, London, UK. (IntelliSys 2016).
  • T. Mittal, L. Bangalore, D. Jayagopi. A Logo-Based Approach for Recognising Multiple Products on a Shelf, published in the SAI Intelligent Systems Conference 2016, London, UK. (IntelliSys 2016).
  • S. Rasipuram, D. Jayagopi. Automatic Assessment of Communication Skill in Interface-based Employment Interviews using Audio-visual Cues, published in the second International workshop on Affective Social Multimedia Computing, IEEE International Conference on Multimedia and Expo (ICME) 2016.
  • K. Shubham, D. Jayagopi. Gesture and Posture recognition using a depth camera with applications in employment interview behavior analysis, IEEE CONECCT 2015, Bangalore.
  • G. GopanK, N. Sinha, D. Jayagopi. Hybrid Features based Classification of Alcoholic and Non-alcoholic EEG, IEEE CONECCT 2015, Bangalore.
  • G. GopanK, N. Sinha, D. Jayagopi. EEG Signal Classification in Non-Linear Frame-work with Filtered Training Data, EUSIPCO 2015, Nice, France.
  • M. Haloi, D. Jayagopi. A Robust Lane Detection and Departure Warning System, Intelligent Vehicles Symposium 2015, Seoul, South Korea
  • M. Haloi, D. Jayagopi. Characterizing driving behavior using automatic visual analysis, poster at the IBM ICare  workshop 2014, Bangalore.
  • B.M. Palavalli, H. Kasi, D. Jayagopi. Analyzing gaming-simulations using video based techniques.,  in the  Conference  on Technological  Ecosystems  for Enhancing  Multiculturality  (TEEM) 2014, Salamanca, Spain.

Teaching

Graduate/Doctoral level

SP 825 – Visual Recognition – Jan to May 2019

DS/NCS 826 – Advanced Visual Recognition – Aug to Dec 2018

DS/NCS/ESD 866 – Advanced Machine Perception – Jan to May 2018

DS/NCS/ESD 866 – Advanced Machine Perception – Aug to Dec 2016

Graduate

CS/DS 612 – Machine Learning – Jan to May 2018

DS/NCS/ESD 863 – Machine Perception – Jun to Jul 2017 [Offered at Samsung – Executive MTech Elective]

DS/NCS/ESD 863 – Machine Perception – Jan to May 2017

DS/NCS/ESD 863 – Machine Perception – Jun to Jul 2016 [Offered at Samsung – Executive MTech Elective]

DS/NCS/ESD 863 – Machine Perception – Jan to May 2016

DS/NCS/ESD 863 – Machine Perception – Aug to Dec 2015

CS/NCE 854 – Digital Image Processing – Jan to May 2015 with Prof. Sinha

CS/NCE 855 – Designing Gaming Simulations – Aug to Dec 2014 with Prof. Sebastiaan Meyer (KTH Sweden)

CS/NCE 854 – Digital Image Processing – Aug to Dec 2014 with Prof. Sinha

Undergraduate

EC304 – Digital Signal Processing – Aug to Dec 2018 

EC304 – Digital Signal Processing – Aug to Dec 2017

ES105 – Signal Processing – Aug to Dec 2016 with Prof. Sinha

ES105 – Signal Processing – Aug to Dec 2015 with Prof. Sinha

ES104 – Signals and Systems – Jan to May 2015 with Prof. Sinha

ES105 – Signal Processing – Aug to Dec 2014

ES101 – Basic Electronics – Jan to May 2014 with Prof. Jyotsna

PGD Artificial Intelligence and Machine Learning

Statistical ML

PGD Data Analytics

Predictive Analytics I


Research & Consulting

  • Accenture Research Grant – Incorporating Multimodal Nonverbal cues in Dialog

  • BEL Project - Design and Development of Deep Learning based traffic analytics modules on infrastructure based cameras

  • SERB(DST) –  Automatic prediction of communication skill in self-presentation interviews, video conference based interviews and group discussions
     
  • Mphasis Center of Excellence Funding – Accessibility for disabled – Indian Sign Language Synthesis, targeting the mute and deaf community

  • CAIR, DRDO – Human Behavior Analysis using Indoor Camera
     
  • NI R&D – Robust and fast Multi-object tracking and FPGA
     
  • SIEMENS – Open Platform for Smart Campus
     
  • IIITB Start up Grant — Automatic Analysis of communicative behavior in job-hiring scenario