For visual effects and animation studios, scaling compute in the cloud is a solved problem. With AWS EC2 and AWS Deadline Cloud, spinning up thousands of render nodes to meet a Friday delivery takes minutes.

Scaling storage access to feed those nodes, however, has remained a major hurdle. When your primary source of truth lives on on-premises NFS filers (or in a separate cloud region), pointing a massive fleet of cloud render nodes back across a WAN connection creates a severe bottleneck. High latency slows down frame times, repeated file reads saturate your AWS Direct Connect or VPN links, and metadata storms (LOOKUP, GETATTR) can bring your local studio storage to its knees impacting on-prem artists.

Previously, studios were forced to choose between manually syncing terabytes of project data to cloud storage before rendering, modifying pipeline paths, or paying for heavyweight managed storage clusters that sit idle between bursts.

KNFSD File Cache on AWS eliminates that tradeoff. Here is a look at how this open-source, AWS-supported architecture works, why industry heavyweights like Wētā FX and Industrial Light & Magic (ILM) are running it in production, and how Integrated Media Technologies (IMT) can deploy and tune it for your pipeline.

What is KNFSD File Cache?

KNFSD (Kernel NFS Daemon) File Cache is an open-source, high-performance NFS caching proxy built on proven Linux kernel technologies (nfs-kernel-server and cachefilesd/FS-Cache) and optimized to run on Amazon EC2 (including AWS Graviton instances).

Deployed inside the same Availability Zone as your cloud render fleet, KNFSD sits in-line between your remote NFS storage and your compute nodes. It mounts your existing on-premises or multi-region NFS exports and re-exports them to your render farm using standard NFS protocols (v3, v4.1, or v4.2).

Two-Tier Local Caching Architecture

Every KNFSD proxy node uses a two-level cache hierarchy to serve read-heavy VFX assets (textures, USD scenes, geometry caches, container images, and tool trees) at local speeds:
  • Level 1 (RAM): The Linux block cache serves ultra-low latency reads directly from instance memory.
  • Level 2 (Local NVMe): When data exceeds available RAM, cachefilesd persists terabytes of cached data onto high-speed local NVMe SSDs.

Only a complete cache miss traverses the network back to your origin filer. Once a single render node pulls a heavy texture or environment map, every subsequent node on that proxy reads it locally at cloud speeds delivering aggregate throughput for less than $6 per hour per 100 Gbps.

Five Major Advantages for VFX Workflows
  1. Zero Pipeline or Client Modifications Because KNFSD uses native NFS re-export semantics without proprietary agents or custom drivers, your render nodes mount the cache just like any standard NFS share. You don’t have to rewrite file paths, alter asset management databases, or rebuild worker AMIs. Writes pass directly back to your origin storage as the single source of truth.
  2. Two-Tier Fan-Out for Massive Compute Fleets For massive burst rendering, KNFSD supports a two-tier fan-out topology. A powerful primary proxy caches directly from your on-prem filer, while an Auto Scaling group of secondary KNFSD nodes scales out horizontally behind a Network Load Balancer or DNS round-robin to serve thousands of concurrent render nodes without overwhelming the origin gateway.
  3. Protect On-Premises Artist Productivity Render farms generate massive volumes of repetitive metadata operations. KNFSD absorbs those metadata lookups and repeat reads at the proxy layer, offloading your on-premises storage controllers so local artists working in Maya, Nuke, or Houdini never experience storage lag during a cloud burst.
  4. Unlock Fearless Spot Instance Rendering Because the cached assets persist independently on the KNFSD tier, studios can aggressively run render fleets on deeply discounted Amazon EC2 Spot Instances. If AWS reclaims a Spot node and spins up a replacement, the new instance immediately pulls from the warm KNFSD cache rather than re-fetching data over your WAN link.
  5. Drastically Lower Networking & Storage Costs Instead of over-provisioning permanent cloud storage or upgrading expensive dedicated interconnect circuits to handle peak render traffic, data crosses your WAN link once. When the render burst finishes, the KNFSD cluster scales down with your compute.

Proven at Global Scale: Wētā FX and ILM

KNFSD File Cache isn’t a theoretical tool it evolved directly out of the extreme production demands of the world’s largest visual effects studios.

Wētā FX

When Wētā FX needed to scale their cloud render farm to 250% the size of their on-premises facility for Avatar: The Way of Water, an early co-developed prototype of this NFS caching system was instrumental in feeding show data to the cloud across extreme distances.

“Our collaboration with AWS on the original KNFSD prototypes proved that high-performance caching at extreme distance is practical for cloud rendering. The areas we identified for improvement after Avatar—elastic scaling, load balancing, and tiered caching—are precisely what AWS has now built into KNFSD File Cache. It productizes exactly the kind of capability we had to build by hand.”

— Kimball Thurston, CTO, & Andy Wright, Head of Pipeline, Wētā FX

Industrial Light & Magic (ILM)

ILM collaborated directly with AWS during the development of the KNFSD Solution Guidance to ensure enterprise-grade observability—exposing critical internal metrics via OpenTelemetry and Amazon CloudWatch so production engineering teams aren’t flying blind during high-stakes deliveries.

“The KNFSD caching solution has surpassed our expectations. During development we collaborated with AWS to ensure the critical internal metrics we needed for production at scale were exposed, saving us from building custom tooling. Thanks to this engagement, we went from a place of uncertainty to having the confidence in now running this solution effectively in production across a number of projects.”

— Greg Newman, Sr. Manager of Production Engineering and Digital Resources, ILM

How IMT Helps You Deploy and Optimize KNFSD

While KNFSD File Cache is open-source and comes with a foundational Terraform module, putting an in-line caching layer into a live production pipeline requires careful architectural tuning. Misconfigured NFS mount options, improper metadata cache timeouts (acregmin/acdirmax), or undersized network interconnects can quickly turn a cache into a bottleneck.

As an AWS Advanced Tier Services Partner specializing in Media & Entertainment, IMT takes the engineering heavy lifting off your plate:
  • End-to-End Architecture & Network Sizing: We evaluate your existing on-premises storage (whether Dell, NetApp, Qumulo, VAST, or Weka) and configure secure, low-latency connectivity via AWS Direct Connect or Site-to-Site VPN tailored to your burst profiles.
  • Turnkey Terraform & Infrastructure-as-Code Deployment: We integrate the KNFSD AMI, Auto Scaling groups, and two-tier fan-out topologies directly into your AWS environment alongside AWS Deadline Cloud or your existing render manager.
  • Pipeline-Specific Cache Tuning: Every studio’s I/O profile is different. We tune kernel re-export parameters, NFSv4.1+ file handle settings, and metadata caching behaviors specifically for your DCC tools and asset structures.
  • Observability & Managed Support: We wire KNFSD’s OpenTelemetry and CloudWatch dashboards directly into your operational monitoring or manage the environment for you so your engineering team stays focused on creative pipeline tools rather than infrastructure plumbing.

As Paul Judkins, VP of Cloud Services at IMT, noted alongside the AWS launch: “An NFS cache is something our customers have asked us for repeatedly over the years. It’s great to have an AWS-supported solution we can confidently put in front of them.”

Ready to Scale Your Render Pipeline Without Migrating Your Storage?

Don’t let storage latency dictate your studio’s render capacity or project deadlines. Contact the IMT Cloud Services team today to schedule an architecture review or set up a proof-of-concept KNFSD deployment in your AWS environment.

For a deeper technical walkthrough of the proxy mechanics and deployment architecture, watch Sean Wallitsch’s presentation on High-Performance Open Source NFS Caching with KNFSD File Cache, which directly breaks down how this AWS solution accelerates visual effects and rendering workflows.