Case study · AI assistant reliability

I tried Grok for its speed. I came away focused on preserving files and methods.

By Loïc Feuga, founder of TechOS. A personal account of my Grok trial: fast responses, files to recover and methods I want to reuse.

I was looking for Dots.
I tried Grok.

01 · Why I tried it

I had been following Dots closely and wanted to use it. At the time of my trial, I could not access it in France. I had contacted OpenAI support several times: I was told the rollout was gradual, with no guaranteed access even on a Business plan. So I set up a daily watch for any change in availability.

Videos presented Grok as an alternative and prompted me to try a promotional offer. That was the starting point for my trial, not a claim that the products are equivalent or a statement about their availability today.

Those Markdown files
were my study material.

02 · The work in progress

My learning process was practical: take a course on its website, extract the text page by page into Markdown files, then generate Quizlet-style revision cards. Those files represented work already done and the material for my next study sessions.

I normally use OpenAI tools heavily every day, including automations. I wanted to see how Grok fitted into real work, beyond a demonstration.

I liked the speed.
The extra guidance, less so.

03 · First impressions

I genuinely appreciated how fast Grok responded, noticeably faster in my experience. But I had to guide it more and answer questions that seemed obvious to me. A quick response does not remove the time spent explaining what I need.

During this trial, I exhausted my weekly quota in about four hours of intensive use.

Grok deleted my files
without warning me.

04 · The breaking point

During the trial, Grok deleted my files without warning me. I had not committed them to Git yet, so there was no commit from which to restore their contents. The issue was the missing commit, not merely a missing push. I take responsibility for not having versioned them yet. That does not excuse deleting them.

I first asked Grok to recover them. It told me it had made a copy the day before using cp on its computer in the cloud. It did restore some files from those copies, but not all of them. I had not asked it to keep them there.

IntelliJ’s local history also helped me recover files. Six were still empty.

Six exports recovered
from extraction artifacts.

05 · The recovery

I returned to Codex, to the chat “Exporter le cours vers Markdown”, and asked it to recover the contents. The JSON captures used for extraction were still present locally in /private/tmp. The chat found them and restored the six exports to their original locations.

The recovery chat checked the text, page order, sources and original sizes. No course was browsed again and no passages were invented. Recovery took a few minutes in the conversation.

The original exports’ limitations remained: images and videos excluded, a linked PDF not transcribed, and the second scenario in 1.2.7 not launched. I had not asked for those artifacts to be retained: they had been produced to extract and check the content. Their presence helped me this time. This was neither magical model memory nor a guaranteed backup feature.

A method must survive
the next chat.

06 · Continuity

I had also refined the tone of some texts in Grok. I then wanted to turn that into a skill: a documented, maintainable and reusable method that the tool can load when needed. Grok told me it did not work with that mechanism or the automatically loaded instruction file I was looking for.

I put together a .grok folder and an agents.md file pointing to my instructions and skills. In every new chat, I had to ask manually for that file and then its instructions to be loaded. Instead of smoothly building on earlier work, I was reintroducing manual steps, repetition and the burden of maintaining continuity.

At TechOS, I believe that for many professional uses, documenting, maintaining and reusing methods through skills is essential to consistency and building on previous work. Having to load them manually in every conversation is a major obstacle to professional use. I am criticizing this limitation of the tool, not the professionalism of people who work differently.

That was the gap I encountered during my trial. A future improvement would not erase the repetition it imposed on me. This account does not claim to describe every feature of every Grok product.

The verified result

Six exports restored. No course reinvented.

The six exports that were still empty regained their original content from local captures. These are the page counts for each restored export.

1.2.4

46 pages restored

1.2.5

49 pages restored

1.2.6

32 pages restored

1.2.7

13 pages restored

1.3.1

37 pages restored

1.3.2

47 pages restored

I choose what preserves
my work.

My choice

I paid for the month and remain open to using Grok for suitable tasks: I really liked its speed. But I prefer a slower, relevant and reliable tool to a fast one that destroys work.

OpenAI remains my current choice based on this experience. That is no promise that one of its tools will never delete anything. My choice rests on concrete criteria: relevance, preserving files, reusing methods and the time I spend guiding or repairing.

The method

Test reliability beyond speed.

The checks I take from this experience when choosing and setting boundaries for an assistant in daily work.

1. Protect existing work

Version useful content before handing changes to an assistant. Keep an independent backup and check how to restore it.

2. Set boundaries for deletion

Define the files in scope and require confirmation before deletion. Review changes: an instruction alone is not a technical safeguard.

3. Test reuse

Document a method as a skill, then open a new chat. Check whether the right instructions are found and loaded without explaining everything again.

4. Evaluate the whole task

Look at relevance, preserved files, reused methods and time spent guiding and repairing. Response speed is only one criterion.

For your team

AI consulting

Choose suitable tools and build methods your team can retain, check and reuse.

Explore our AI services

Let’s talk

Does your AI preserve what you build?

Let’s discuss your uses, your files and the methods you want to reuse every day.

Let’s talk about your project