AiRWARDS 3.0
Developed the chat screen of an AI service on the defense network with 1,000+ DAU and 5,000+ MAU. Reduced repeated rendering and screen lag in long responses.
Role
On a defense network AI service with 1,000+ DAU and 5,000+ MAU, handled chat as one of the 2 frontend developers on an 8-person product team. Led the user-focused definition of the initial 3.0 features, the UX and the design of the state management structure, and developed the entire chat screen. Implemented a PDF viewer, code blocks, custom weather and news components, and source footnotes inside answers.
Problem
On defense network PCs with about 4GB of RAM, scrolling and input lagged as conversations grew longer, and this got worse after the switch to GPT-OSS raised TPS and made rendering more frequent. Analysis with Chrome DevTools and React Profiler found a bottleneck: for every SSE token, react-markdown re-parsed the whole response and re-rendered even finished code highlighting and tables.
Solution
- BufferingCollected SSE tokens in a buffer and applied them in batches, separating the token receiving cycle from the rendering cycle
- Pinned blocksDid not re-render finished code and table blocks, and updated only the block being written
- Lighter loadApplied virtual scrolling to long conversations and replaced the code highlighter with a lightweight library
Result
Measured with Chrome DevTools on a defense network PC with about 4GB of RAM, using the same response with a lot of code and tables, main-thread work time went from about 800ms → 80ms (about 90% reduction)
Overview
AiRWARDS is an AI service specialized for the Air Force and used on the defense network, with 1,000+ DAU and 5,000+ MAU. For the move to 3.0, handled the frontend development that combined the question answering, previously split into general, military and legal modes, into a single chat screen.
Handled chat as one of the 2 frontend developers on an 8-person product team. Led the user-focused definition of the initial 3.0 features, the UX and the design of the state management structure, and developed the entire chat screen. Also took part in developing the document summarization and speech recognition features.
Other internal service work during Air Force service is written up in Air Force experience.
Key features
- AI question answering: Questions and answers in general, military and legal fields on a single chat screen
- Components inside answers: PDF viewer, code blocks, custom weather and news components, source footnotes
- Document summarization: Summarizes the key points of a document
- Speech recognition: Converts speech to text
Implementation and improvements
-
Finding the cause of the repeated work through profiling
On defense network PCs with 4GB-class RAM, scrolling and typing got slower as a conversation grew longer, and this was worst in long answers with code highlighting and tables. After the switch to GPT-OSS, tokens per second (TPS) went up, rendering happened more often, and the lag got worse. The Chrome DevTools Performance recording showed main-thread work piling up, and the React DevTools Profiler showed components rendering repeatedly.
Server-sent events (SSE) is a way for the server to send a generated answer in real time, token by token. Each time a token arrived, React state changed, and
react-markdownparsed and rendered the whole answer being generated again. In the process, syntax highlighting and table handling for code that was already complete were repeated too. The aim was to keep frequent updates while an answer is generated and still reduce the repeated work, so the update frequency and the area to redraw were each adjusted. -
Collecting tokens to reduce screen updates
Updating the screen on every token lets people see the answer being generated more often, but Markdown parsing and rendering are repeated just as often. Changed the code to collect tokens received over SSE in a buffer and apply them in batches, which reduced the number of state updates. With this approach the screen is not updated on every token, so it balances frequent updates of the answer against processing cost.
Receive token → Accumulate in buffer → Apply in batch → Update screen -
Separating the area being generated from completed blocks
Split the answer into the area being generated and the content blocks that are already complete. New tokens go into the area being generated, while the rendered output of completed code blocks, tables and diagrams is fixed so that they are not processed again. Buffering reduced the number of updates, and the separation narrowed the range reprocessed in a single update.
New answer ├ Text being generated (updated) ├ Completed code block (fixed) ├ Completed table (fixed) └ Completed diagram (fixed) -
Virtual scrolling for long conversations, and a lightweight code highlighter
The longer a conversation gets, the more elements have to be drawn. Applied virtual scrolling to long conversations so that mainly the visible part is drawn. Also replaced the code highlighter with a lightweight library to reduce the load on PCs with 4GB-class RAM.
Tech stack
React · TypeScript · Vite · Tailwind CSS · shadcn/ui · React Query
Spring Boot
Results and current status
Measured with Chrome DevTools on a defense network PC with 4GB-class RAM, using the same response with a lot of code and tables, main-thread work time dropped from about 800ms to about 80ms. That is a measured reduction of about 90%.
By applying tokens in batches and not reprocessing completed blocks, the repeated frontend work during answer generation was reduced.
AiRWARDS presented at the Air Force booth
The booth where the Air Force's in-house AI system was presented. Service screens and guide materials were displayed together.