BAN CARS
BAN CARS. Using state-of-the-art VLMs and human-in-the-loop realtime feedback, help people reimagine their cities free of cars. Cityscape segmentation + inpainting ➡ ✨magic✨
YouTube Video
Project Description
BAN CARS is an augmented reality web app that helps people explore different futures for the built environment. The human is at the center of the augmented reality loop, and is able to select how to reimagine their surroundings. The flexibility of this approach allows for arbitrary restyling of the surroundings on demand, to allow the user to select everything from walkable urban realism to mountains in the shape of the nearby buildings. The frame rate of multiple frames per second over a mobile network allows the user to feel in control of the transparent process and see how the augmented reality accommodates their updating views. Modifying our cities to be safe for children and to respond to the climate crisis is one of our greatest challenges of the century, and if people can view a better future they can fight for it.
- functionality: segmentation, multi-modal models, and more, in a tight time budget
- innovation: widely available web app, user-customizable prompt for repainting
- real-world impact: help people reimagine their environs to make the city safer for their children and more pleasant to get around and ready for decreased carbon budgets
- human-in-the-loop: you can customize the prompt to reimagine the city as you want to (“cyberpunk”, for example)
- working code: React web app, Python + huggingface H100 server, &c
- collaboration: help the AI help you reimagine your city, so you can fight politically for it to be different
- unique: realtime visual feedback, in the future we can run the entire stack locally on-device (maybe a two day project further)
- trust and transparency: realtime visual feedback helps people understand how their prompts affect their output
Prior Work
I’ve used React and webcams in the browser before, I’ve used many of these models before. The core of the app, transforming the picture through the prompt within about a hundred milliseconds on the server, was all created during the hackathon. The viewer, a react app with a variety of options and customizations, was adapted from other augmented reality react apps I’ve made, and was under a hundred lines of code
Team
Technologies
Products & Tools
Additional Links
Try BAN CARS yourself! It's an augmented reality web app, so it works best if your video camera can point to something that looks like a streetscape.