Community Live 14: AI on your terms – Using local LLMs with Claris FileMaker
The session was centered on securely integrating AI with FileMaker applications by running local LLMs to manage and perform semantic search capabilities. Many attendees were interested in using local models to address concerns over data security when integrating AI and managing sensitive data. A key goal was to show how to set up and operate these AI models locally, providing attendees with the knowledge to implement and maintain secure, locally-hosted AI solutions that keep data completely within their network.
Step-by-Step Guide for Setting Up a Local LLM for FileMaker
1. Understanding the File Structure and Transferring Files
- Accessing Required Files: FileMaker Server installations across different operating systems (Windows, macOS, and Ubuntu) include a folder containing open-source Python scripts and models necessary for embedding functionality.
- Windows: Located in
C:\Program Files\FileMaker\FileMaker Server\Tools - macOS: Found inside the
.dmginstaller package, typically underExtras - Ubuntu: Inside the FileMaker Server zip installer ./Open_Source_LLM/
- Windows: Located in
- Purpose of Files: These Python scripts serve as the foundation of a microservice (a lightweight web server) that runs the embedding model.
- File Transfer: Move these files to a separate, high-performance machine that is not the FileMaker Server. Running the AI model on the FileMaker Server itself is discouraged because AI models can be CPU- and memory-intensive. Running the model on the FileMaker Server would risk resource conflicts, potentially degrading the server’s performance and user experience.
2. Security Configuration: SSL Certificates and PKI Authentication
- Why Security Matters: Since the AI model functions as a microservice accessible over the network, securing it with SSL (Secure Socket Layer) encryption is essential for protecting data in transit, especially for sensitive or confidential information.
- SSL Certificate Setup:
- Obtaining an SSL Certificate: Obtain an SSL certificate from a trusted certificate authority (CA) or use a self-signed certificate for internal development purposes.
- Environment Variable Configuration: Define environment variables to tell the microservice where to find the SSL certificate and its key. For instance:
SSL_CERT_FILEfor the certificate pathSSL_KEY_FILEfor the key file path
- Platform Differences: On Windows, environment variables can be set via the Control Panel, while on macOS or Linux, they can be set via command line or shell configuration files.
- Port Settings:
- Default Port: The microservice listens on port
8080by default, as specified in thefm_LLMOS_Settings.pyfile (with theSERVER_PORTvariable). - Custom Port Configuration: To change the port, update the
SERVER_PORTvariable in the settings file. Ensure this port is accessible on your network and not blocked by firewalls, as this is essential for secure communication between FileMaker and the microservice.
- Setting Up Public Key Infrastructure (PKI) for Authentication:
- PKI secures the microservice further by ensuring that only authenticated users can send requests to it. This setup protects the service from unauthorized access or potential malicious requests.
- Generating PKI Key Pair: Create a private-public key pair:
- Private Key: Retain this on the server hosting the AI model.
- Public Key: Share this with FileMaker clients to verify the JSON Web Token (JWT) for authenticated access.
- JWT (JSON Web Token): A JWT generated with the private key is used as a token for FileMaker script steps. When FileMaker interacts with the microservice, it presents this token to authenticate the request.
- Environment Variable for Public Key: Set an environment variable (e.g.,
PKI_KEY_FILE) pointing to the public key on the microservice machine to enable verification of JWTs.
3. Python Environment and Dependency Management
FastAPI Framework: The Python-based FastAPI framework is used to create the REST API microservice, enabling HTTP/HTTPS communication between FileMaker and the model.
Using Conda for Isolated Environments:
Purpose of Conda: Conda is used to create an isolated environment, which helps avoid conflicts with system-wide Python dependencies.
Environment Creation:
Set up a virtual environment (e.g., named webinar_live) and install Python within it.
Install additional Python packages using pip, specifically the jwt library for token handling.
Using requirements.txt File: FileMaker provides a requirements.txt file listing all necessary packages. Use the following command to install these dependencies:
<code>pip install -r requirements.txt</code>This step downloads and installs all necessary libraries in the Conda environment, such as packages for embedding, model handling, and FastAPI.
FileMaker Integration: Key Script Steps and Settings
1. Script Steps for Configuring and Performing Semantic Searches
- Configure AI Account:
- This script step allows users to set up an endpoint for accessing the AI model. Select the “Custom” AI provider and specify the endpoint URL (IP address or DNS name) and any necessary authentication, including the JWT.
- Embedding and Semantic Search Script Steps:
- Insert Embedding: This step sends data (text or images) to the model to create a vector embedding, which is then stored in FileMaker for future searches.
- Perform Semantic Find: Searches are performed by comparing embedding vectors for a text description or using an existing image’s embedding as a query.
2. Available Models and Custom Model Configuration
- Default Models: FileMaker 21.1 includes models like
clip-vit-base-patch32, suitable for image and text embeddings, with different versions for monolingual and multilingual use. - Adding Custom Models:
- Users can add custom models from repositories such as Hugging Face by specifying the model name. If the model isn’t pre-installed, the system will prompt the user for download permission.
- Considerations for Model Selection:
- Model Size and Speed: Larger models can deliver more accurate results but are more resource-intensive.
- Domain-Specific Models: For specialized applications (e.g., medical images), select models pre-trained on relevant data to ensure accurate embeddings.
3. Security Configurations in FileMaker Script Steps
- Token Authentication:
- Include the JWT in the FileMaker
Configure AI Accountscript step’s API key field to authenticate calls to the microservice, ensuring only authorized users can access the model.
- Include the JWT in the FileMaker
- HTTPS Endpoints:
- When configuring the AI endpoint, ensure that the URL begins with
https://for secure data transfer. Using DNS names for endpoints is recommended as they can be aligned with SSL certificates for improved security.
- When configuring the AI endpoint, ensure that the URL begins with
# the setup
Configure AI Account [ Account Name: "twentyonezero" ; Model Provider: Custom ; Endpoint: "https://192.168.2.206:8080" ; API key: "your_api_key_here" ]
// Alternative DNS-based endpoint configuration (commented out):
// Configure AI Account [ Account Name: "_AI_account" ; Model Provider: Custom ; Endpoint: "https://wim.ets.fm:4443" ; API key: "_AI_jwt" ]
# encoding the image
Insert Embedding [ Account Name: "twentyonezero" ; Embedding Model: "clip-ViT-L-14" ; Input: PHOTO::thumbnail_picture ; Target: PHOTO::embedding_container ]
// Alternative embedding model (commented out):
// Insert Embedding [ Account Name: "twentyonezero" ; Embedding Model: PHOTO::embedding_model_gt ; Input: PHOTO::thumbnail_picture ; Target: PHOTO::embedding_container ]Best Practices for Deploying and Maintaining Local LLMs
1. Hardware and Resource Allocation
- Dedicated Server for AI Models:
- Run the model on a separate machine dedicated to handling embeddings, especially in production environments, to prevent resource competition with FileMaker Server.
- Consider using a machine with high CPU and GPU power if performing frequent or complex embeddings. GPU-accelerated machines, like high-end Apple or Windows developer laptops, are well-suited for model testing and development.
- Network Configuration:
- Reverse Proxy: Use a reverse proxy server, such as NGINX, to route requests securely to the model. This setup enhances security by isolating the model from direct internet access and provides centralized management of endpoints.
2. Ensuring Persistent Operation
Daemonizing the Microservice:
<code>pm2 start "python path/to/start_server.py" --name "AI_Model_Service" --env production</code>Monitoring: pm2 provides monitoring features that allow users to track memory and CPU usage, restart the service automatically if it crashes, and schedule regular restarts.
3. Setting Up a Development Workflow for Testing and Optimization
- Testing the Model: Before deploying to production, test the model in a development environment. Monitor memory usage, query response time, and overall system performance.
- Separate Development and Production Models: Run separate instances for development and production to prevent potential conflicts or downtime during testing.
Demonstrated Use Cases for Local LLMs in FileMaker
1. Image Embedding and Semantic Search
- Purpose: The session demonstrated how to use embeddings to perform natural language queries on images. For instance, searching with a description like “colorful abstract” or “birds on a wire” retrieves relevant images based on their visual content.
- Process: Both the query and images are converted into embeddings, which are compared to find the closest matches.
- Storage: Embeddings can be stored as vectors within FileMaker fields for later retrieval. These vector representations allow quick similarity searches without repeatedly embedding the same images.
2. Find Similar Functionality
- Script Step for Similarity Search:
- The
Find Similarfunction uses the embedding of a selected image as a query to search for similar images in the database. This method bypasses the need for natural language input, relying instead on direct image comparison.
- The
- Use Cases:
- Cataloging: This feature is ideal for product cataloging, where users may want to find visually similar items.
- Content Management: Organizations can use this functionality to organize and retrieve images based on visual characteristics rather than relying solely on metadata.
3. Domain-Specific Models for Specialized Data
- Medical Images: For fields like medical imaging, using specialized models pre-trained on relevant datasets improves accuracy.
- Extracting Text from PDFs:
- For text-heavy PDFs, extracting text and running it through a text embedding model is advised. This approach enhances search accuracy for documents with text content.
Claris Community Resources and Future Sessions
- Claris Community and Documentation: Claris has published extensive blog posts, community discussions, and resources covering topics such as SSL, PKI, and AI model setup. The Claris Community is highly active, with numerous discussions on model selection, security practices, and troubleshooting.
- Upcoming Webinars:
- Developer-Focused Enhancements: Future sessions will delve into developer-focused improvements in FileMaker 2024.
- Expanded AI Capabilities: An additional webinar will cover FileMaker’s new features for image semantic search and enhanced server performance and stability.
Final Results and Takeaways
This session provided a detailed roadmap for deploying and managing secure, local AI models in FileMaker, equipping participants with the knowledge to independently set up and maintain this integration for secure, efficient semantic searches.
Wim decorte shared these resources during his presentation to help you configure a local LLM.
Engineering blog: Working with LLMs in FileMaker 2024 – the step by step instructions Wim followed are in the Configure Open Source LLM section. All the necessary files for setup are included in the Claris FileMaker Server 2024 installation.
Python resources because that’s what the Local LLM microservice is written in using the FastAPI framework
- Get started using Python on Windows for beginners
- FastAPI A free tool to set up the Python web service
- Python environment management with miniconda
- PM2 Process Management
Resources for SSL certificates and PKI token set up
- Microservice to generate the private/public key pair and the JWT token
- Instructions to set up the key pair included in this article
- SoliantMike’s FM Admin API tool
- Engineering Blog: Using PKI authentication for FileMaker Server Admin API calls
Community Live 13: Jumpstart AI in Claris FileMaker recording step by step demo of how to configure semantic search in your FileMaker apps.
