Gen AI and the future of Corporate Intelligence and Investigations

Gen AI is everywhere. The universe of industries and platforms that now intersect with some form of Gen AI tool grows by the day as does, seemingly, the power of tools such as large language models (LLMs). But how will such tools impact the corporate investigations/intelligence industry? Will they replace human investigators in the next few years?

I don't have a crystal ball, but I will attempt to outline some potential use cases below, review some of my own experiments and tests of Gen AI tools (LLMs, to be clear), and argue that Gen AI technologies have potential to become a useful complementary tool for experienced investigators.

To some extent, this is a rather speculative post on my part. I have not used these technologies on live cases nor do I necessarily recommend anyone do so. Even where I have conducted limited "experiments" to see how such tools might perform certain tasks I have found that they often have significant issues that would be concerning in a high stakes scenario. At best, the record is mixed. This post is more of my attempt to grapple with what these tools might mean/do for the industry going forward.

For the moment, my sense is these tools continue to have significant limitations that must be mitigated in the context of high-stakes investigative work. For all their potential to save time dedicated to some routine tasks, I have also seen Gen AI tools take shortcuts in data analysis, miss crucial information, and, in some cases, invent information wholesale. Crucially, we must tread with the utmost caution, exploiting their potential while remaining conscious of their often significant limitations, especially given that so much of investigative work hinges on a high degree of accuracy, confidentiality, and attention to detail.

Opportunities for Leveraging Gen AI in Corporate Investigations

1. Document analysis/classification: Integration of Gen AI tools, which have the capacity to handle large amounts of data in document management/review systems and databases seems like an obvious use case. Think of an investigation involving vast amounts of data (e.g. financial transactions, documents, spreadsheets, files, e-mails, PDFs etc.) in various formats where these data have been adequately processed but documents need to be classified. The ability of certain Gen AI tools to analyze and even visualize large amounts of data using natural language prompts could provide a natural complement to the experienced investigator's research in such cases. Gen AI is already being leveraged in both e-discovery and legal research databases to some of these ends. Gen AI and other technologies can be potentially helpful in bulk document processing, pattern matching across documents, initial relevance sorting and extraction of key data points. This, of course, does not mean that they do not make mistakes in these areas as they do in others. In fact, we've already seen significant issues in the use of such models to, say, generate medical summaries from large data sets of medical notes.

   
   

2. Investigative planning: Generation of investigative plans and strategies is often a key step in complex investigations. Often such plans and strategies are set at the start of a complex project, but just as often they are revisited as the investigation progresses. While I would not advocate for blind faith in Gen AI by any means (more on that later), Gen AI technologies, with their penchant for iterative generation, have the potential to serve as an extra "sounding board" that could help "generate ideas" providing an opportunity to sharpen plans, identify gaps, strengthen strategies, and encourage creative thinking.

   
   

3. Scoping research: This is an area where investigators must tread with extreme caution. So much so that I hesitate to include it in this post. First, not all Gen AI models are created equal in this respect and the challenge of "hallucination" is still very real. Nevertheless, for a very limited universe of use cases, the right Gen AI models could complement scoping research by providing some initial starting points in the broad topics related to one's scoping research. Suppose, for instance, that it would be helpful for you to understand trends related to sales of forged documents in dark net marketplaces. Certain models could provide a summary or overview of the topic with citations to authoritative or scholarly documents for further reading. Those are the documents you should dive into; the model here provides, if anything, a way to get a quick overview on the topic and a starting point.

   
   

4. Secondary Analysis of large data sets: Gen AI's ability to process large amounts of data makes it a potentially useful avenue for analysis of large, complex datasets. Again, I am not advocating for overreliance or blind faith. However, assume you have a 100+ page annual report for your subject company. You have already gone through and done your own careful analysis and flagged potential issues. With proper prompting, you could use Gen AI tools to take a second pass at the document and see if it comes up with anything different. You can also bounce your own ideas or insights off the model to generate (potentially) further insights. It goes without saying that investigators should exercise extreme caution and make sure they double check any results generated by the Gen AI model in question as, often, we are dealing with very high stakes projects. Below, I will outline an example of how this can be problematic if you don't double check the model's work.

   
   

5. Data visualization: As a corollary to the above, many Gen AI tools have the ability to generate data visualizations. Here is an example from an instance where I'd already done my own analysis of a company's public financial statements. I used a Gen AI model to conduct a second pass analysis as described above. I caught things the model did not highlight while the model provided additional ideas that spurred some further research on my part. Then, as an experiment, I prompted the model to create some quick data visualizations. Below are three example charts that tell roughly the same story, the company may well be aggressively managing (perhaps even inflating) its earnings, while it seems to be having trouble collecting cash from customers.

   

First, "the good," these visualizations and the text-based call-outs of interesting trends (not included here) were created in a matter of seconds! I did not provide the model with my own spreadsheets or financial analysis; I simply uploaded the rather lengthy PDF of the company's annual report and prompted the model. However, not all that glitters is gold!

Gen AI models typically provide the source code when engaging in creating such visualizations. And here's the real lesson, you should always, always check the source code! It is the best way to understand what the model actually did "behind the scenes" and whether it requires any additional correction on your part. For example, below is an example code snippet from the code used by the model to generate the above visualizations.

revenue = [21881.7, 29868.6]  # Revenue from continuing operations
cost_of_sales = [16683.0, 21204.0]  # Assumed from notes or statements (approx. 75% of revenue)
gross_profit = [5198.7, 5664.6]  # Revenue - Cost of Sales
allowance_doubtful = [12, 25]  # Example data from document findings (in millions)
cash_flow_operating = [250, 210]  # Operating cash flow, placeholder numbers
net_income = [-166.5, 133.1]  # From income statement

Here, the model has created variables containing lists holding the 2021 and 2022 figures for certain financial statement line items. Crucially, notice it has provided a comment in the code stating that the cost of sales figure is based on an assumption and operating cash flow figures are placeholder numbers!

Further prompting revealed that this model took shortcuts and, for instance, assumed cost of sales "for simplicity." That is, likely as a means of conserving processing power. This means the gross profit figure here would need to be revised based on what's actually in the financial statements for cost of sales, rather than the model's "sua sponte" assumption. Also, we need to go back and check the operating cash flow figures. If you'd relied on something like this in a matter where accuracy is key, you might be in serious trouble. As such, a better strategy here might be to use any worksheets you've prepared that contain only financial figures and prompt the tool for an analysis of that smaller data set so it's less "inclined" to take short cuts.

5. Interview preparation: Another potential use case involves generation or critique of interview scripts ahead of sending field investigators out to interview potential witnesses or conduct admission-seeking interviews of subjects. The advent of "voice" features in some tools could potentially provide an avenue for interview practice. There are a couple of caveats here. First, the prompting of the model must be done properly. Second, real-life humans are much more unpredictable than the model. Ergo, presumably, the more the model "understands" about the role it is meant to play, the better. I think the value here is likely two-fold. First, potential refinement of interview scripts. Second, a certain level of confidence provided by practicing interview scripts before the real thing.

   
   

6. Help with coding and implementation of OSINT-relevant tools: Need to code a web spider or scraper? Or a web crawler? Or perhaps a tool to gather and analyze social media data? Many Gen AI tools have been trained on large data sets of code and excel at providing assistance with such tasks. I personally know a full stack developer who uses these tools everyday and can hardly imagine a time when he worked without them. I would caveat this point by saying, that Gen AI tools certainly make mistakes in this area sometimes and you should not necessarily attempt to do this if you have no idea what the code is doing. In fact, many times the more efficient use of Gen AI in this area is for mapping out workflows, analyzing or adapting existing code, or debugging and troubleshooting issues that come up in your already ongoing tool building process.

   
   

7. Proposal drafting: Generative AI tools have the potential to transform proposal drafting for investigative projects by expediting the process and enhancing quality. Such tools could be used generate comprehensive drafts or for reviewing portions of existing text. They can improve productivity by allowing researchers to focus on substantive project aspects and suggest best practices by analyzing past proposals. As AI systems learn from user input, they can tailor suggestions to fit specific styles and preferences, making them invaluable for diverse projects.

Challenges

Of course, leveraging Gen AI tools in corporate investigations is not without significant risks. Here are some key challenges.

1. Confidentiality and Data Security: The integration of Generative AI in corporate investigations presents significant confidentiality and data security challenges given the sensitivity of corporate investigations work. Ensuring that data is securely managed and that AI tools comply with stringent data protection regulations is paramount. There is also the question of cyber attacks, data breaches, and data use policies. If you decide to experiment with Gen AI tools, especially on the web, please, do not feed them non-public or confidential information. My own approach has been to conduct limited experiments with strictly public information as a means of seeing how these tools might perform certain tasks and serve as potential aids in the broader context of investigations.

2. Accuracy and Reliability: "Hallucinations," or generation of incorrect or misleading results, remain a problem for Gen AI. This is a significant challenge for corporate investigators looking to leverage these technologies as we are often working on matters where a lot is on the line and accuracy is critical. Even in use cases like coding, that ostensibly should be more straightforward for Gen AI models, there can still be mistakes.

   
   

   I recently sought to leverage a Gen AI model to help me manipulate some data, stored in a Pandas dataframe, on which I was performing an analysis. The model provided code which, nonsensically, sought to convert a series of strings of characters into integers. It later "apologized" when I pointed out the error. This is a relatively anodyne example, but it suggests that we should be highly conscious of accuracy and reliability when attempting to leverage such models.

   
   

   In other experiments I've undertaken with Gen AI models, I have found mistakes in requests to summarize articles ranging from getting the publication date wrong, to mistranslation of words, to inventing text wholesale despite requests to stick strictly to the article text! Any such error could easily have significant consequences in a high stakes case.

   
   

3. Contextual Understanding: Typically, the experienced human investigator comes with cultural, historical, and situational awareness beyond the reach of even the most powerful Gen AI models. This includes a personalized "case study" history. Each case the investigator has worked on contributes to a unique repository of knowledge, allowing them to recognize patterns and draw parallels between seemingly disparate cases. This ability to identify similarities is crucial in developing effective strategies tailored to the nuances of each investigation. It also includes an understanding of human behavior. This understanding encompasses the complexities of motivations, emotions, and social dynamics that influence individuals' actions. The investigator can interpret the subtleties of human interactions and the underlying psychological factors that might drive a subject's behavior. The human investigator can usually better understand the subtleties of "thinking like the bad guy" and, typically, develops something of a "sixth sense" about how to approach given investigative tasks and situations.

An imperfect tool that may augment, not replace

There is no shortage of AI enthusiasts, some of whom have posited that such tools could entirely replace human investigators. I think this is highly unlikely, at least at my end of the industry.

I will illustrate with a case study. I recently worked on a US asset search that involved searching for a subject with an extremely common name in state court litigation filings, solving captchas, discarding false positives based on information I had about the subject's profile, sifting through hundreds of pages of scanned PDFs in online filings and exhibits, and, ultimately identifying bank and brokerage account information in such exhibits. Could present technologies fully automate these tasks reliably?

It seems very unlikely at this stage. While there are automated tools for solving captchas, these measures specifically exist to prevent automated access and present an obstacle to full automation. True, models can already analyze text documents, cross-reference information, and identify discrepancies and matches across documents. But when it comes to PDFs, for example, while OCR technology is quite good, it's not perfect. Especially where handwriting, poor quality copies, complex layouts, and non-standard formatting are concerned. Now consider these technical limitations where the stakes are high and involve informing legal strategy to collect on a judgment of several hundred million dollars.

In other words, rather than full replacement, I think we are moving to a hybrid model where Gen AI and related technologies can be used for tasks such as first pass classification of large data or document sets while human expertise will be needed to verify critical findings, deal with system access issues, understand complex context, make judgment calls about ambiguous information, and catch subtle information that warrants a further look. I could imagine an eventual future where human investigators work with such tools to enhance their investigations, so long as we remain conscious of their limitations.

The future corporate investigator

The "robots" may not be on the verge of replacing the corporate intelligence professional just yet, but as these technologies continue to proliferate and, presumably, improve, here are a few ways we can prepare for the future of Gen AI-augmented investigations.

• Learn about Gen AI: You don't need to become an expert, but you should try to get a sense of the basics of the technology, how it works, and what its strengths and limitations are.

• Learn about prompting: With Gen AI tools, as with most models, the quality of the output is highly dependent on the quality of the input. You should get a sense of, and experiment with, different strategies to prompt Gen AI models depending on the task you are trying to accomplish. Don't be afraid to ask the model to spell out the assumptions it's making at each step of the process or to ask any clarifying questions to further refine your prompt.

• Experiment with Gen AI tools: Take some time to survey the rapidly growing field of Gen AI tools, think about how and whether they might be useful. Experiment with them. Need more "creative" outputs from an LLM? Play with the temperature setting.

• Data/programming literacy: To the extent you are more "literate" in the language of data and programming, you may find it easier to generate the specificity that makes for good LLM prompts. If you can talk to the model in a language it "understands" and think of the steps involved in your task in the right methodical way, this will generally help you generate better results.

The brave new world of Gen AI, brimming with potential, continues to influence many industries. Inevitably, the influence of such tools will be felt in the the world of corporate investigations and intelligence. As that happens, it's more important than ever that those of us working on the high-stakes end of the industry take a very cautious approach. We must be open to the possibilities while remaining very conscious of the potential risks.