Leveraging AI and Model Frameworks for Designing AI Applications
In the rapidly evolving landscape of generative AI, building and designing AI applications can seem daunting. Staying ahead requires not only understanding the latest models but also knowing how to utilize them effectively. While tools such as Azure AI Foundry and CoPilot Studio make it easier to build and see the capabilities of AI and agentic applications, they often abstract away critical AI mechanics necessary for designing and understanding AI solutions at a deeper level.
Despite the excitement surrounding AI, investing in it for the sake of investing is not a sound strategy. The process should begin with a clear understanding of our business strategy. Are we aiming to enhance customer satisfaction? Improve operational efficiency? Importantly, once we define these objectives, we cannot simply apply AI to existing business processes. We must rethink and reinvent these processes with AI.
A crucial yet often overlooked aspect of this process is defining AI and model frameworks. From my experience, categorizing AI use cases into distinct types provides a structured approach to AI design because the world of AI is not just about LLMs and Gen AI. One effective method I like to utilize is a “Task Based” Framework that classifies our use cases into six categories: Expansion, Compression, Conversion, Seeker/Retrieval, Tool/Function, and Reasoning.
Expansion Prompts
Expansion prompts are among the most common LLM use cases, generating content by expanding on a simple input. Models such as GPT-3.5, GPT-4, or Gemini 1.5 can take a brief input and generate detailed outputs. Examples might include asking CoPilot to provide a detailed explanation of quantum computing, asking ChatGPT to write you a song about the beach, or asking an LLM to take a simple marketing idea and expand it into a full campaign for fitness tracking.
Compression Prompts
Compression prompts are the opposite of expansion prompts in that they distill large inputs into concise outputs. This is particularly useful for summarizing lengthy documents, extracting key points from research papers, or generating meeting summaries. Models like Claude 3.5, Llama 3.3, and GPT-4 excel in providing quick and accurate summaries.
Conversion Prompts
Conversion prompts transform information from one format to another, such as translating languages, converting code between programming languages, or reformatting data. For instance, if I want to convert a natural language query into a Python statement, I may favor a specialized model like Codestral-2501, that is specifically designed for code generation and review tasks and is fluent in over 80 programing languages. Further, while LLMs such as GPT4o are well equipped for language translation, if I have a very specific use case such as translating between English and Japanese, I might pick NTT’s tsuzumi model, which is specifically trained to be highly effective for English-Japanese conversions.
Seeker/Retrieval Prompts
Seeker or retrieval prompts focus on locating specific information within larger datasets, making them invaluable for document searches, information extraction, and pattern recognition such as querying a sales report to identify the best-performing product in a specific quarter or finding details about a car part number in a manual. This could also be used in a scenario where I want to scan through security logs and write a prompt such as, “Find all instances of failed login attempts from unknown IPs in the past 24 hours.” Models such as GPT-4o, Claude 3.5, and Gemini 2.0, which offer multimodal support, are particularly well-suited for these types of enterprise search, document analysis, and Retrieval-Augmented Generation (RAG) integrations.
Tool/Function Prompts
Tool or function prompts, execute commands and trigger workflows, making them fundamental for tool integration and function calling. For this type of use case, the LLM would analyze my input and determine if an external tool or function is needed to answer the request. For example, if I generate the prompt and ask my AI Agent, “What is the current weather in Orange County, California?” the LLM could generate a JSON output that calls my tool api like this:
{
"tool_name": "get_current_weather",
"arguments": {
"location": "Orange County, California",
"unit": "fahrenheit"
}
}The external weather tool will receive this message, execute using the defined arguments, and return the results to the LLM in order to give me a response.
GPT-4o and Claude 3.5 are commonly used for such tasks due to their strong interaction capabilities with client-side tools. However, for more cost-sensitive applications, open-source models like Cohere’s Command R+ offers a viable alternative for multi-step tool use cases.
Reasoning Prompts
Reasoning prompts facilitate decision-making by providing judgments, conclusions, and insights. These are useful for planning, problem-solving, risk assessment, and recommendation systems. For example, an RFP evaluation tool can leverage reasoning prompts to perform complex analysis by evaluating the accuracy and validity of its response through link and fact checking. Or if I was looking at medical data, an example prompt might be, “Given the following symptoms, lab results, and patient history, what are the possible diagnoses?” Models like the new GPT-4o1, Claude 3.5, and Gemini 2.0 perform well in this domain, and their effectiveness can be improved further by implementing techniques such as chain-of-thought (CoT) prompting or agentic frameworks like the maker-checker model.
Final Thoughts
Organizing AI application uses cases into these categories can help bring clarity to development efforts. Frameworks like this can improve decision-making by narrowing down prompt engineering strategies and selecting appropriate tools for each use case. Additionally, they facilitate reusable methodologies and tooling, making AI engineering more systematic and efficient.
I have found using this framework acts as valuable step when designing AI solutions. By categorizing prompts into Expansion, Compression, Conversion, Seeker/Retrieval, Tool/Function, and Reasoning, AI practitioners can streamline workflows and enhance decision-making when building their applications. As the generative AI ecosystem continues to evolve, I believe leveraging structured frameworks will be crucial for staying ahead and maximizing the potential of the latest advancements.
If there is a particular framework or approach you find effective, I’d love to hear about it!
Until next time!