Many language models can process more than just text—they can analyze images, PDFs, and other files. The AI SDK provides built-in support for sending files to your LLM provider's API.
Right now, your chat application has a file upload button on the frontend, but it doesn't actually do anything with the file. The backend is expecting only text messages, so uploading an image won't work.
You need to modify the form submission handler to capture the uploaded file and send it along with the user's text message to the LLM.
fileToDataURL helper function already provided in your codeThis function converts a File object from the form into a string that the AI SDK can send over the network.
onSubmit callback in the ChatInput componentInstead of passing only text to sendMessage(), you need to pass a parts array that includes both a text part and an optional file part.
Look at the message parts documentation to understand the structure you need to create.
Use the fileToDataURL function to convert the file to a data URL string. Include the file's mediaType property so the LLM knows what kind of file it is.
sendMessage() call to use a parts arrayThe parts array should always include a text part with the user's input. If a file was selected, add a file part to the array as well.
Look at the solution code to see what the FileUIPart type looks like.
pnpm run devOpen http://localhost:3000 in your browser.
image.png file from the problem folderThis is an image of Lake Bled in Slovenia.
Type "Could you describe this image?" in the chat input
Submit the form and check that the LLM successfully analyzes the image
The model should describe what it sees in the image rather than failing silently.
Gemini 2.5 Flash has this capability built in.
const fileToDataURL = (file: File) => {return new Promise<string>((resolve, reject) => {const reader = new FileReader();reader.onload = () => resolve(reader.result as string);reader.onerror = reject;reader.readAsDataURL(file);});};
onSubmit={async (e) => {e.preventDefault();const formData = new FormData(e.target as HTMLFormElement,);const file = formData.get('file') as File | null;// TODO: figure out how to pass the file// _as well as the text_ to the// /api/chat route!// NOTE: You have a helpful function below// called fileToDataURL that you can use to// convert the file to a data URL. This// will be useful!// NOTE: Make sure you handle the case where// `file` is null!sendMessage({// NOTE: 'parts' will be usefultext: input,});setInput('');setSelectedFile(null);}}