What an AI engineer (LLM) does
An AI engineer builds applications on large language models (LLMs), whether those come from a model provider or are open-weight models you host yourself. Rather than training models from scratch, they design the system around the model: retrieval-augmented generation that grounds answers in your documents, prompts and tool calls, agents that carry out multi-step tasks, evaluation suites that measure answer quality and guardrails that keep outputs safe. They then ship and operate the application like any other production software.
A chatbot demo is quick to build; a system your customers and staff can trust is not. Without careful evaluation, LLM applications give confident wrong answers, expose data or cost far more to run than planned. Finch screens candidates for LLM applications they have put in front of real users, how they measured quality and how they handled privacy, security and cost, so you meet engineers who build carefully as well as quickly.
Typical responsibilities
- Design retrieval pipelines, including document chunking, embeddings, vector search and reranking
- Write and version prompts, tool definitions and agent workflows
- Build evaluation sets and automated tests that score accuracy, grounding and tone
- Add guardrails such as input filtering, output checks, permission controls and human review
- Deploy and monitor LLM services, tracking latency, failures and usage costs
- Compare hosted and self-hosted models on cost, privacy and answer quality
Qualifications and credentials
- A degree in computer science, software engineering or a related field, or equivalent professional experience
- Solid software engineering skills in Python or TypeScript, including APIs, testing and version control
- Hands-on experience with LLM provider APIs, vector databases and orchestration frameworks such as LangChain or LlamaIndex
- Experience designing evaluations and handling problems such as hallucination and prompt injection
- Understanding of Canadian privacy law such as PIPEDA and of data residency questions with cloud AI services
Who hires an AI engineer (LLM)
- SaaS companies adding AI features to existing products
- Banks, insurers and fintech firms building customer service and internal knowledge tools
- Law, accounting and consulting firms working with large document collections
- Health technology companies, where privacy and accuracy carry extra weight
- AI startups building products on top of language models
Where we recruit AI engineer (LLM) candidates
We fill AI engineer (LLM) roles for employers across Toronto and the GTA, including North York, Scarborough, Etobicoke, Vaughan, Markham, Richmond Hill, Mississauga, Brampton, Oakville, Burlington, Oshawa and Pickering, and for employers across Canada. GTA employers can meet us at our North York office by appointment.
What Finch checks before you meet a candidate
- LLM applications they shipped to real users, and what changed after launch
- How they built evaluation sets and decided an answer was good enough to release
- Retrieval design choices, and how they diagnosed poor or ungrounded answers
- Security habits, including defences against prompt injection and data leakage
- Awareness of running costs, and of when a smaller model would do the job
Interview questions to ask
- Our assistant sometimes cites the wrong policy document. How would you find out why and fix it?
- How would you test a prompt change before it reaches customers?
- When would you advise us against an agent and recommend a simpler fixed workflow instead?