A Comprehensive Guide by Honza Koudelka and Johan Hedman
Table of Contents
- Introduction to FileMaker Performance Challenges
- Understanding Multi-Threading and CPU Utilization
- Impact of Mixed Client Connections on Performance
- Identifying and Mitigating FileMaker’s Bottlenecks
- Optimizing Temporary File Management
- Implementing Proxy and Caching Solutions for High Traffic
- Best Practices for FileMaker Backup Strategies
- Managing WebDirect and Data API Traffic
- Upcoming Architectural Changes in FileMaker
- Building a Scalable FileMaker Solution
- Acknowledgments
1. Introduction to FileMaker Performance Challenges
As organizations continue to scale up their FileMaker deployments, optimizing performance becomes crucial for preventing bottlenecks. Performance limitations in FileMaker can significantly impact workflows, causing slowdowns that frustrate users and reduce productivity. This post brings together expert insights from Honza Koudelka and Johan Hedman at FMTraining.TV, focusing on strategies to identify and resolve performance issues in FileMaker solutions.
2. Understanding Multi-Threading and CPU Utilization
FileMaker Server operates on multiple specialized engines, each with unique tasks, such as script processing, custom web publishing (CWP), and data APIs. However, multi-threading in FileMaker Server has been an ongoing challenge, as each engine often operates independently without sharing workloads across CPUs. Despite early attempts to enable true multi-threading with FileMaker Server 18, Claris reverted to a single-threaded database engine, which has limited certain CPU-intensive tasks.
Key Points on Multi-Threading and CPU Utilization:
- Single-Threaded Engines: Most FileMaker Server engines, like the database engine and script engine, operate on a single CPU at a time.
- CPU Load Distribution: Each service (e.g., Data API, script engine) assigns tasks to only one CPU, which can cause individual CPUs to max out under heavy use, even on powerful hardware setups.
- Impact on Scaling: With a high CPU load from one service, FileMaker can experience significant slowdowns, even with advanced hardware.
Expert Tip: To address these limitations, ensure sufficient CPU allocation and monitor CPU load distribution closely to avoid performance bottlenecks due to high CPU usage.
3. Impact of Mixed Client Connections on Performance
When deploying solutions with a mix of FileMaker Pro clients, WebDirect users, and Data API connections, performance can degrade due to varying loads and resource consumption by each connection type. FileMaker handles requests differently based on the client type, with API requests and WebDirect sessions requiring more complex data handling than direct Pro connections.
Observations on Client Type Impact:
- Concurrent Connections: Mixed client environments (e.g., FileMaker Pro clients and WebDirect users) place varied loads on the server, potentially creating a bottleneck for other clients.
- API Load Sensitivity: API connections, especially those making frequent requests, can degrade performance for other FileMaker Pro users due to the added processing strain.
Best Practice: Monitor the distribution of client types accessing the server, and consider segregating API traffic or implementing load-balancing techniques to mitigate performance issues.
4. Identifying and Mitigating FileMaker’s Bottlenecks
Koudelka and Hedman emphasize the importance of top call statistics and CPU usage per service as key diagnostic tools for identifying FileMaker bottlenecks. The “spinning wheel” or “coffee cup” icon indicates a bottleneck, often caused by intensive calculations, heavy data loads, or complex layout rendering.
Strategies for Mitigating Bottlenecks:
- Monitoring with Top Call Statistics: Regularly check the top call statistics log to identify slow operations.
- Optimizing Layouts and Calculations: Simplify layouts and avoid complex calculations within views to reduce server load.
- Using the Admin API for Idle Session Management: Disconnect idle sessions proactively, especially CWP sessions, as these can linger and consume resources.
Actionable Tip: Use the Admin API to disconnect idle sessions to reduce unnecessary server load, particularly for Custom Web Publishing users.
5. Optimizing Temporary File Management
FileMaker creates temporary files for each user session, caching data to improve response times. However, temp files can accumulate, becoming an issue for large deployments if left unchecked. Koudelka and Hedman suggest regularly purging these files to prevent slowdowns due to temp file buildup.
Temporary File Management Insights:
- Session Caching: Temporary files store layout and query data, reducing load times for returning users.
- Purge Schedule: Set up a script to delete unnecessary temp files, especially after maintenance windows or releases, to ensure system responsiveness.
Maintenance Tip: Regularly clear temporary files to avoid performance degradation in large deployments, particularly those with high concurrent users.
6. Implementing Proxy and Caching Solutions for High Traffic
For solutions experiencing high traffic, such as e-commerce or large data applications, proxy caching can alleviate server load by handling repetitive requests outside the FileMaker Server. Hedman’s custom proxy solution caches requests and redistributes them across engines, ensuring efficient data retrieval for high-demand applications.
Proxy and Caching Strategy:
- Custom Caching by Request Type: Cache static data (e.g., product prices) for longer periods, reducing server calls for frequently accessed but rarely updated information.
- Load Distribution: A proxy server can route different requests to separate engines (e.g., Data API vs. CWP) to prevent overloading any single service.
Actionable Tip: Use a proxy server to cache frequently accessed data, thereby reducing FileMaker Server load and improving response times for high-traffic environments.
7. Best Practices for FileMaker Backup Strategies
Effective backup scheduling is essential but can impact performance if done too frequently. Progressive backups, which only store changes since the last backup, reduce server load and speed up the process. Koudelka and Hedman recommend scheduling backups during low-traffic hours to minimize their impact on server performance.
Backup Strategy Recommendations:
- Progressive Backups: Utilize progressive backups to capture only changed data, reducing backup times and system strain.
- Scheduled Backups During Low-Traffic Hours: Avoid scheduling backups during peak hours to minimize their impact on system availability.
Key Insight: For large deployments, configure progressive backups and schedule full backups during off-peak hours to reduce server load while ensuring data integrity.
8. Managing WebDirect and Data API Traffic
WebDirect and Data API requests can significantly impact performance, particularly in high-traffic scenarios. Hedman recommends implementing a proxy pool that queues and sequentially processes requests, preventing overload and ensuring reliable access.
WebDirect and Data API Best Practices:
- Request Queueing: Use a proxy pool to manage high volumes of simultaneous API calls, reducing server load and ensuring stable performance.
- Concurrent User Monitoring: For solutions with more than 50 concurrent WebDirect users, consider additional worker machines to distribute the load effectively.
Actionable Tip: Implement a proxy pool to handle high WebDirect and Data API traffic, particularly during peak times, to prevent system overload.
9. Upcoming Architectural Changes in FileMaker
Claris has announced plans to improve FileMaker’s architecture, introducing OData as a direct connection to the FileMaker database engine, bypassing the XML API. This upgrade promises faster data retrieval and reduced latency for integrations, along with the introduction of Nginx as the primary web server across all platforms.
Notable Upcoming Changes:
- Direct OData Connection: Claris plans to connect OData directly to the database engine, which should improve data access speeds significantly.
- Nginx Web Server Integration: Nginx will replace the current web server for improved stability and scalability, which is expected to benefit WebDirect and Data API performance.
Key Insight: Claris’ architectural changes, such as the direct OData connection and the integration of Nginx, will provide FileMaker with improved scalability and response times.
10. Building a Scalable FileMaker Solution
Optimizing performance in FileMaker requires a strategic approach to resource allocation, monitoring, and traffic management. Koudelka and Hedman’s insights underscore the importance of regular system maintenance, intelligent backup scheduling, and effective load distribution for high-traffic deployments. By adopting these best practices, administrators can ensure robust performance for FileMaker solutions, regardless of scale or complexity.
Final Takeaway: FileMaker’s architecture presents unique performance challenges, but with careful monitoring and strategic optimizations, organizations can build scalable, high-performance solutions that meet the demands of their users.
11. Acknowledgments
This post is based on insights shared by Honza Koudelka and Johan Hedman at FileMaker Training Videos, who have greatly contributed to the FileMaker community through their expertise in performance optimization and system architecture. Their findings provide invaluable guidance for developers and administrators managing large-scale FileMaker deployments.
