Architect

Job summary

Extends the generative AI surface of Flowsource inside the client environment building Retrieval Augmented Generation RAG pipelines grounded on the clients own codebases knowledge bases and engineering standards configuring Code Companion developing prompt libraries and operating the Flowsource VS Code extension Evaluates LLM output quality against client coding conventions governs GitHub Copilot or equivalent integration and instruments Gen AI adoption metrics across the engineering org

Responsibilities

Key Responsibilities

Design and deploy RAG pipelines grounded on client codebases architecture docs ADRs runbooks and standards

Configure Flowsource Code Companion and the Flowsource VS Code extension for delivery teams tune retrieval chunking and reranking for the clients content corpus

Build and curate a client specific prompt library aligned to delivery patterns code generation refactoring test scaffolding code review assistants

Govern coexistence with GitHub Copilot or equivalent so the developer experience is coherent not competing

Run output quality evaluations coding standards conformance hallucination rate security PII leakage checks

Instrument Gen AI adoption metrics prompt volume acceptance rate token consumption per developer productivity proxies

Stay current on LLM choice for the client Bedrock Azure OpenAI on premise models and manage cost latency quality trade offs

Required Skills and Experience

5 plus years software engineering with at least 1 to 2 years building production RAG Gen AI systems

Hands on with LangChain or LlamaIndex vector databases eg pgvector OpenSearch Pinecone Azure AI Search

LLM API depth on at least one of AWS Bedrock or Azure OpenAI familiarity with GCP Vertex AI helpful

Strong Python comfort with TypeScript for VS Code extension extension points

Evaluation literacy building eval harnesses golden sets hallucination and bias testing

Awareness of enterprise AI governance data residency prompt logging PII handling

Flowsource Tooling and Tech Stack

Flowsource Code Companion and Flowsource VS Code extension

LLM APIs AWS Bedrock Azure OpenAI primary GitHub Copilot integration surface

Frameworks LangChain LlamaIndex

Vector DBs pgvector OpenSearch Elasticsearch Pinecone Azure AI Search

Python primary TypeScript extension surfaces

Success Metrics

Active developers using Code Companion weekly monthly

Prompt acceptance rate against baseline

Hallucination quality eval score on standing test set

Gen AI token consumption efficiency tokens per accepted output

Number of curated client validated prompts in the active library

Provide your feedback on BizChat