Data Analysis and NLP with LLMs

Data Analysis and NLP with LLMs

Introduction

Language models can be used to perform sophisticated analyses of both textual and non-textual data. This tutorial explores the different ways to use LLMs for data analysis and natural language processing.

Data Analysis with LLMs

1. Structured Data Analysis

import pandas as pd
from openai import OpenAI
import json

class DataAnalyzer:
    def __init__(self, api_key):
        self.client = OpenAI(api_key=api_key)
    
    def analyze_dataframe(self, df, analysis_type="general"):
        # Convert the DataFrame to a statistical description
        stats = df.describe().to_json()
        sample = df.head().to_json()
        
        prompt = f"""
        Analyse les données suivantes :
        
        Statistiques : {stats}
        Échantillon : {sample}
        
        Type d'analyse demandée : {analysis_type}
        
        Fournis :
        1. Un résumé des tendances principales
        2. Des insights clés
        3. Des recommandations d'analyse supplémentaires
        """
        
        response = self.client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}],
            temperature=0.3
        )
        
        return response.choices[0].message.content