Can AI help make the web more accessible?

Building Ally: Using LLMs to redesign websites

At the NY GPT/LLM Hackathon Cornell Tech, @n_sri_laasya, @mrannanay, and I built Ally: an AI-powered web app that makes use of LLMs to generate code to make websites more accessible.

The problem: often accessibility is deprioritized and neglected on websites. In general — what if websites could meet you where you’re at? What affordances can AI provide to browsing the web?

Here we’re thinking about improving sites for people with visual impairments, but you can also imagine design improvements taking into account neurodiversity. How might the experience improve for readers with ADHD?

Given a url, Ally serves back the improved website. Note: we’re not changing the original site, rather, the tool provides a layer to redesign the site. Let the web have a multitude of designs, while also giving readers customization tools. Here’s a few examples.

Improvements including colors and text size:

Improvements including contrast and font size:

The updated CSS includes information on the improvements:

• Add ARIA label attribute for better screen reader support
• Add ARIA role attribute for better screen reader support
• Increase heading font-size for better readability
• Increase contrast for better visibility

We’re using GPT4 to read the html and css and output a website with improvements towards accessibility. The challenge we faced was the limited context window. We worked around this by testing on smaller websites and also seeing how far we could get only reading the css.

There’s still a long way to go, and pumped we were able to demo the idea. Perhaps this sparks other ideas! One feature I’m interested in is using image-to-text models to add missing alt text for images.

We wouldn’t have been able to do this without the time, space, support, resources provided by the hackathon. Super thanks to @davederiso, @agihouse_org, @volokuleshov, @cornell_tech, @LererHippeau. Post adapted from this thread.