Modeling Crowd Behaviors: From Representation to Understanding and Generation

Abstract

Modeling crowd behaviors is useful for emerging technologies like autonomous driving and virtual/augmented reality, and essential for understanding social phenomena such as stress and evacuation analysis. However, it poses significant challenges due to the inherent diversity and indeterminacy of human behaviors. For example, people may choose to turn left or right to avoid obstacles, making this problem infeasible. This dissertation addresses these challenges by presenting a holistic pipeline for crowd behavior modeling. Specifically, we introduce a data-driven approach to accurately capture and simulate crowd dynamics across three key aspects: Representation, Understanding, and Generation.

The first part of this dissertation introduces how to construct the feature representations of the relationships between agents in scenes. The agents consider their surrounding environments, objects, and nearby agents when determining their routes toward destinations. For this, there have been data-driven manners, but the complexity of capturing their relations grows exponentially while managing an increasing number of elements in dynamic, real-world scenarios. To mitigate this, we propose novel techniques to reduce the complexity of the representations in both spatial and temporal dimensions. To be specific, we propose group-based methods, following hierarchical steps that perceive scenes at collective levels and propagate this feature to the individual levels. Next, the agents' motion pattern-based methods allow the recognition of long-term pedestrian behaviors without any need to encode every individual footstep. These proposed methods enable more efficient and tractable context recognition.

The second part of this dissertation focuses on understanding realistic crowd dynamics based on the perceived information. As usual, humans potentially sample the hypotheses for their destination and then select one reasonable route for them. Because of this manner, a future trajectory for each agent should be modeled with probabilistic natures. In computer vision field, it is well-known that generative models work well in handling stochastic tasks. Due to their inherent randomness, conventional generative models struggle to cover the feasible trajectories. One of the proposed methods makes use of human thinking prior, embedded in the pre-trained language model. Additionally, another method, the conditional generative model, infers trajectories step-by-step in a cascading fashion, allowing for realistic predictions even in challenging conditions. These approaches facilitate more realistic crowd dynamics modeling and accurate path planning by understanding human dynamics in scenes well.

The third part of this dissertation explores generating crowd behaviors in simulation spaces. In real world, people have certain walking patterns, going to their destination from starting point in consideration of their spatial layouts. We observe that people sometimes form walking groups as time goes by. To imitate this behavior, we present a learnable crowd emitter, which enables continuous generation of human dynamics. The proposed method accounts for key initialization factors, including agent attributes, starting and destination coordinates, and pace. In addition, locomotion trajectories are planned to guide them toward their destinations as well. Here, we propose a learnable sampling technique that replaces the random sampling process with a purposive manner to ensure diversity in the trajectories. This approach is good at simulating diverse crowd scenarios.

Lastly, the dissertation summarizes the contributions and outlines future directions for further improving data-driven approaches to crowd behavior modeling.