Automated Documentation Generation Procedure with bito.ai

Overview

This document describes the procedure for automated documentation generation using the bito.ai tool, with a focus on using CLI and automation scripts. The tool enables efficient and standardized creation of technical documentation, facilitating the ongoing maintenance and updating of documents using Generative Artificial Intelligence.

Requirements

• bito.ai: The bito.ai tool must be installed and configured in the development environment.
• CLI (Command Line Interface): The CLI must be accessible for executing commands.
• Automation Scripts: Scripts used for automating the documentation generation.

Installation and Configuration

Installing bito.ai

1. Download and Installation:

   • Download the appropriate installer for your operating system from the official bito.ai website.
   • Run the installer and follow the on-screen instructions to complete the installation.
   • For documentation generation, a Linux Ubuntu operating system was used due to the ease of utilizing some automation scripts already provided by the community.

2. Initial Configuration:

   • After installation, configure bito.ai with your credentials and personal preferences.

   • Run the initial configuration command:

     bito config --setup

Creating bito.ai Credentials

1. Account Creation:

   • Visit the bito.ai website and sign up to create an account if you do not have one.
   • Complete the registration process and check your email to activate your account.

2. Generating an API Key:

   • Log in to the bito.ai control panel.
   • Navigate to the API Settings or Credentials section.
   • Click on Create New API Key.
   • Provide a name for the key and select the necessary permissions.
   • Copy the generated API key and store it in a secure location. You will need this key to configure the CLI.

3. Configuring the CLI with the API Key:

   • Run the CLI configuration command with the API key:

     bito configure --auth --api-key YOUR_API_KEY

   • Replace YOUR_API_KEY with the copied API key.

Automation Scripts

Automation scripts were used for documentation generation in two different scenarios: one providing documentation for code and another offering an overview of functionalities based on modules. Below is the explanation of the operation of each. The solution was based on: Bito AI Automation

Automated CODE Documentation Generation Script

Overview

This Bash script automates the creation of documentation for code files in a specific directory using the bito tool. It generates Markdown documentation files for scripts with .sh, .py, .php, and .js extensions, organizing them into a directory structure corresponding to the source directory.

Script Operation

The script follows these steps to generate the documentation:

1. Setting Up Variables

The script sets up some essential variables:

• BITO_CMD: Locates the bito command in the system.
• BITO_CMD_VEP: Defines an additional option for the bito command, depending on the version.
• BITO_VERSION: Retrieves the current version of bito and compares it with version 3.7 to set BITO_CMD_VEP.

BITO_CMD=`which bito`
BITO_CMD_VEP=""
BITO_VERSION=`$BITO_CMD -v | awk '{print $NF}'`
# Compare BITO_VERSION to check if it's greater than 3.7
if awk "BEGIN {exit !($BITO_VERSION > 3.7)}"; then
    BITO_CMD_VEP="--agent create_code_doc"
fi

2. Argument Verification

The script checks if a directory name has been provided as a command-line argument. If it is not provided, the script displays an error message and exits.

if [ $# -eq 0 ]; then
    echo "Please provide folder name as command line argument"
    exit 1
fi

3. Verifying and Creating the Documentation Directory

The script checks if the provided directory exists. If it does not exist, it displays an error message and exits. It creates a documentation directory by prefixing "doc" to the original directory name.

folder=$1

if [ ! -d "$folder" ]; then
    echo "Folder $folder does not exist"
    exit 1
fi

doc_folder="doc_$(basename $folder)"

if [ ! -d "$doc_folder" ]; then
    mkdir -p "$doc_folder"
fi

4. Generating Documentation

The script finds all files with specific extensions in the provided directory and its subdirectories. For each file found, it creates a corresponding directory structure in the documentation directory. It generates documentation for each file using the bito command and saves the documentation in the appropriate directory.

find "$folder" -type f -name "*.sh" -o -name "*.py" -o -name "*.php" -o -name "*.js" | while read file; do
    # Get relative path of file from folder
    rel_path="${file#$folder/}"

    # Get directory path of file in document folder
    doc_dir="$doc_folder/$(dirname "$rel_path")"

    # Create directory if it does not exist
    if [ ! -d "$doc_dir" ]; then
        mkdir -p "$doc_dir"
    fi

    # Create documentation using bito and save it in document folder
    file2write="$doc_dir/$(basename "${file%.*}").md"
    echo "Creating documentation in: " $file2write
    # The below command does not work and gives the following error
    # Only "-p" flag is applicable for this command. Remove any additional flags and then try again.
    # bito -p docprmt.txt -f "$file" >> "$file2write"
    cat $file | bito -p ./prompts/structured_doc.txt $BITO_CMD_VEP > $file2write
done

5. Finalization

After generating documentation for all files, the script displays a message indicating where the documentation has been created.

echo "Documentation created in $doc_folder"

6. Usage

To use the script, run the following command, replacing <folder_name> with the name of the directory that contains the code files:

./script_name.sh <folder_name>

Automated OVERVIEW Documentation Generation Script

Overview

The generate_docs.sh script is designed to automate the generation of high-level documentation for a codebase. It uses the Bito CLI tool to generate textual documentation and Mermaid diagrams for visual representation. The script handles the creation of individual module documents, generates an overview diagram, and aggregates documentation into a comprehensive file.

Prerequisites

1. Bito CLI - A tool for generating documentation.
   • Installation:
     • macOS/Linux: sudo curl https://alpha.bito.ai/downloads/cli/install.sh -fsSL | bash
     • Arch Linux: yay -S bito-cli or paru -S bito-cli
     • Windows: Download the MSI from Bito's website.
2. Mermaid CLI - A tool for generating Mermaid diagrams.
   • Installation: npm install -g @mermaid-js/mermaid-cli

Variables

• BITO_CMD: Path to the Bito CLI command.
• BITO_CMD_VEP: Optional flag for Bito CLI based on version.
• log_file: Log file for token usage information.
• prompt_folder: Directory containing prompt files for NLP tasks.
• total_input_token_count: Total input tokens used in the session.
• total_output_token_count: Total output tokens generated in the session.
• lang_csv: CSV file with programming language extensions.
• skip_list_csv: CSV file containing paths to skip.

Functions

bito_response_ok()

Checks if the response from Bito CLI is valid. Returns 0 if valid, 1 otherwise.

update_token_usage()

Updates the global token usage counters based on input and output text.

log_token_usage_and_session_duration()

Logs the total token usage and session duration to the log file.

check_tools_and_files()

Checks for the presence of required tools and files, exits with an error if any are missing.

read_skip_list()

Reads the skip list from a CSV file to determine which paths should be excluded.

is_skippable()

Determines if a path should be skipped based on predefined patterns.

call_bito_with_retry()

Calls the Bito CLI with retry logic in case of temporary failures.

create_module_documentation()

Generates documentation for an individual module, including Mermaid diagrams.

extract_module_names_and_associated_objectives_then_call_bito()

Extracts module names and objectives from a file and generates documentation using Bito.

fix_mermaid_syntax()

Fixes syntax issues in Mermaid diagrams.

fix_mermaid_syntax_with_bito()

Uses Bito to fix Mermaid syntax issues.

validate_mermaid_syntax()

Validates the syntax of a Mermaid diagram.

fix_and_validate_mermaid()

Applies syntax fixes and validates Mermaid diagrams, optionally using Bito for additional fixing.

generate_mermaid_diagram()

Generates Mermaid diagrams from markdown files.

create_mermaid_diagram()

Creates Mermaid diagrams for a module with retry logic and syntax validation.

generate_mdd_overview()

Generates an overview Mermaid diagram by combining all .mdd files in the specified directory.

create_find_command()

Creates a find command to locate files with specific extensions for documentation.

Execution Flow

1. Check Tools and Files: Ensures all required tools and files are available.
2. Folder Validation: Verifies that the provided folder exists.
3. Documentation Generation:
   • Creates documentation for each module file found.
   • Aggregates individual module documents into a main file.
4. Overview Generation: Generates a comprehensive overview diagram of the system.
5. System Introduction and Summary: Extracts and prepends an introduction and summary to the aggregated documentation.

Usage

./generate_docs.sh <folder_to_document>