FDE and LLM Applications: The Complete Guide (2024)
FDEs play a vital role in the development of large language models (LLM) applications. This article covers everything you need to know about LLM applications and how FDEs build and deploy them
What Are LLM Applications?
A large language model (LLM) is an AI capable of learning patterns in natural language. LLMs can perform various language tasks, such as reading, writing, summarizing, or answering questions.
LLMs are useful, but an LLM application is necessary to perform specialized tasks. For example, applications based on LLM can be used for
customer service,document search,summarizing meetings and emails,code assistance,andgenerating reports.
In practice, each application will be different and require different instructions, data, and design, despite using the same model.
What Is a Forward Deployed Engineer?
A forward deployed engineer (FDE) is a type of software engineer that focuses on understanding customers’ problems and developing solutions. Some FDEs work on-site while others are remote but always communicate closely with the client.
This type of engineer emerged from Palantir but is now in demand in many AI companies and start-ups.
Why Do LLM Applications Need FDEs?
Building an example application is simple — any user can try it by writing a prompt and receiving a response. In contrast, a production-grade application is considerably more involved. It usually contains sensitive data that requires appropriate security measures. Moreover, the application must provide accurate responses even to unexpected queries. Finally, FDEs are needed to integrate the application within a company’s existing infrastructure.
Most LLM applications fail at the stage of proving useful to the user. FDEs help to overcome this challenge by constantly interacting with the customer and studying their requirements.

How Do FDEs Develop LLM Applications?
The process of developing an LLM application is iterative and may involve several stages.
1. Needs Analysis
FDEs begin their work by understanding what problem the application should solve. They analyze what their customers are currently doing and what automation they want to achieve. At this stage, FDEs also determine the expected outcome and response format. This information helps to avoid wasting time and resources on unnecessary model training. Once the goal is identified, FDEs select the most appropriate model for the task.
2. Data Preparation
LLMs require a lot of data to train, but FDEs often work with limited resources. Moreover, this data must be prepared and formatted in a certain way. FDEs need to analyze what information the application should contain and how it should be structured. They also study who should have access to what data. Sometimes FDEs deal with confidential information, so this step is the most complicated one.
3. Model and Prompt Selection
LLMs can be very large, but it is necessary to choose the most cost-effective one for the task. FDEs determine the required model parameters and select the most optimal ones for the project. They also develop prompts to fine-tune the model and constantly improve them. This iterative process allows the model to work better with specific requests.
4. Retrieval
A trained model usually does not have access to internal data. FDEs solve this problem by implementing a retrieval process. It means that the application will also contain a search system that will scan documents and extract the most relevant information for the model. This method is often called RAG, which stands for retrieval-augmented generation. It improves the quality of responses and helps to verify their accuracy.
5. Testing and Evaluation
FDEs cannot deploy the application immediately after training the model. They need to evaluate how the LLM works in practice by testing it with real queries. FDEs usually select the most common questions and measure the model’s accuracy, reliability, and response time.
At this stage, FDEs can also measure the model’s performance against competitors. Finally, they determine whether the application achieves the desired quality within acceptable cost and time budgets.
6. Launch and Iteration
When the model passes all tests, FDEs can deploy it. However, their work does not end here as they need to constantly improve the application. They analyze user queries, collect feedback, and fix application errors. Over time, FDEs add new features to the application based on customer needs.
An Example of an LLM Application
A law firm wants to create an application that would allow its employees to quickly find the necessary clauses in contracts. An FDE can examine this process and identify inefficiencies. They can structure the contract base and develop an application that finds the necessary information and even summarizes it. The FDE can test this application by working with actual questions from lawyers. Thus, the lawyers will receive responses in less time, and the model itself will constantly improve.
Challenges of Implementing LLM Applications
LLM applications have several common pitfalls that FDEs need to be aware of. First of all, such models may give incorrect answers with confidence. Second, applications often work with confidential information that must be kept secure. Third, FDEs need to consider the cost since these models can be unprofitable.
Besides, users do not want to wait for responses for a long time, so FDEs need to optimize the model. Finally, customers need to trust the application, and the FDE must convince them of its relevance, effectiveness, and reliability.
Skills for FDE and LLM Application Jobs
There are several competencies that are critical for FDE and LLM applications jobs. First of all, technical skills are in demand among FDEs. They need to know how to work with Python, TypeScript, and SQL. Moreover, they must understand how to implement API, data, and pipeline design. Finally, FDEs need to know how to work with vector databases, retrieval, and prompt engineering.
For LLM applications, programming languages such as Python and SQL are also required. FDEs need to know how to train and evaluate models, as well as design prompts. Working with cloud platforms such as AWS, Azure, and Google Cloud is also advantageous.
Finally, soft skills are also important for FDEs since they deal mainly with people. Communication skills, patience, responsibility, and a willingness to constantly learn and adapt to new conditions are advantageous for this job. In summary, both technical and soft skills are critical for the position of FDE.
Why Are FDEs in Such Demand?
FDEs are in demand for several reasons. First, the need for applications based on LLM is growing, but many companies do not have the expertise to develop them. Second, enterprise customers want FDEs to support their products.
Third, data privacy and compliance are becoming more important, so companies need FDEs to ensure security. Fourth, product teams need FDEs to help them understand their customers’ needs.
Consequently, knowledge of FDEs increases the qualifications of any software developer.
Careers as an FDE
A career as an FDE is advantageous for several reasons. First, FDEs can move into product management, engineering leadership, or consultancies. Moreover, the most experienced FDEs can provide their own expertise to start their own companies. Finally, the FDE role serves as a springboard to a variety of careers in AI.
How to Get Started
Anyone can start a career as an FDE by learning Python and SQL. They also need to study how LLM works and how to create effective prompts. To gain experience, one can try creating a simple RAG application that finds answers to questions in documents. It is also advisable to add tests to check the correctness of the results. Moreover, it is important to develop the skill of explaining things to non-experts.
Finally, one should learn enough about a particular industry to advise customers. Attending training courses can help acquire new skills for an FDE position. Most importantly, however, is the practical application of knowledge. Anyone can start a successful career as an FDE with the necessary skills and motivation.
Final Thoughts
FDE positions and LLM applications are in great demand. Although models are often the focus of attention, applications are what bring value and quality to the user. FDEs develop such applications and understand the audience’s needs.
As these engineers have a wide range of skills from programming to interacting with customers, this type of specialist is in demand in the modern world. Consequently, an FDE is a great choice for anyone looking to work in the exciting field of AI.