国际新闻 · 中文译文与原文对照

星期五 最新 1,000 篇 · 滚动浏览

科技

Uber将翻译缓存内存减少了多达90%,同时仍保持P99延迟在1毫秒以内。

来源:InfoQ 技术发布 上传时间 (多伦多时间)1 家来源 · 1 篇报道

中文译文

Uber已重新设计基于Go语言的翻译缓存机制,将初始内存占用降低了90%以上,同时保持p99缓存未命中延迟在1毫秒以内。该混合架构将翻译键及频繁访问的数值存储在内存中,而将其余数值存储在磁盘上。该系统每秒可处理数百万次翻译请求,并保持99.9%以上的缓存命中率。

此次优化是为了解决通过内存分析在Uber的Go语言服务中发现的两个问题。在某些服务中,垃圾回收操作占用了20%以上的CPU资源,而翻译相关内容约占所有已分配对象的70%。

Uber工程团队指出:

在Uber这样的规模下,即使是在内存管理方面存在微小的效率问题,也会累积成较大影响。第一次优化是用由RWMutex保护的常规map[string]string结构替换了Go语言中的sync.Map。该系统的工作负载以读取为主,更新操作很少,且没有单独的删除操作。之前的实现方式为每个翻译条目创建多个对象和指针,从而增加了垃圾回收的开销。改为基于值的映射结构后,减少了对象分配,在相关服务中使CPU利用率提升了30%以上。

然而,减少对象分配并未解决整体内存消耗问题。翻译数据集可能包含两百到三百万个键,尽管单个服务通常只访问其中的一小部分。缓存刷新又带来了另一个问题:该服务会在后台构建一个替代缓存,然后再与现有缓存进行原子替换。因此两个缓存会同时占用内存,导致某些服务在刷新过程中出现内存不足的情况。Uber曾考虑过外部缓存以及生成针对特定服务的翻译子集,但这两种方法都会带来额外的运营或维护成本。因此,该团队采用了混合存储设计,保留了现有的Get()和Put() API。所有键都存储在内存中以便快速查询,而翻译内容则存储在基于磁盘的文件中。

Uber的混合翻译缓存(来源:Uber博客文章)

该实现使用了256个分片,每个分片都有独立的读写互斥锁、内存中的条目以及磁盘文件。基于六字节哈希值的键可降低元数据开销,当发生冲突时则回退到原始键格式。基于时间的全局最近最少使用(LRU)策略用于管理内存中的条目。在缓存刷新时,系统会利用访问历史记录预加载过去15分钟内被访问过的翻译内容,从而避免缓存刷新后出现空缓存的情况。在一个示例中,此前在内存中占用约800 MB空间的翻译数据集,在重新设计后仅需要约40 MB用于存储元数据。在设定50,000条内存存储条目的限制下,实际内存占用通常在80 MB到120 MB之间。在另一项测试中,超过400万条条目中有约30,000条被加载到内存中,而缓存命中率仍保持在99.9%以上。由磁盘支持的缓存未命中情况的p99延迟时间则低于100微秒。

内存使用量从约2.7 GB降至750 MB(来源:Uber博客文章)

Uber的实践凸显了大规模内存缓存的成本影响。该团队建议尽量减少对象引用以降低垃圾回收的开销,并监测缓存数据的访问频率。当只有少量缓存数据被频繁使用时,可采用混合方案,将频繁访问的数据保留在内存中,而将其余数据存储到磁盘上。

原文

Uber has redesigned its Go-based translation cache to reduce its initial memory footprint by more than 90% while keeping p99 cache miss latency below one millisecond. The hybrid architecture retains translation keys and frequently accessed values in memory while storing the remaining values on disk. The system serves millions of translations per second and maintains a cache hit rate above 99.9%.

The optimization addresses two problems identified through memory profiling across Uber's Go services. Garbage collection (GC) consumed more than 20% of CPU utilization in some services, with translations accounting for approximately 70% of allocated objects.

Uber Engineering noted that

At Uber's scale, even small inefficiencies in memory management can add up.

The first optimization replaced Go's sync.Map with a conventional map[string]string protected by an RWMutex. The workload was predominantly read-heavy, with infrequent updates and no individual deletions. The previous implementation created multiple objects and pointers for each translation entry, increasing GC overhead. Moving to a value-based map reduced object allocations and improved CPU utilization by more than 30% in the affected service.

However, reducing object allocations did not solve the overall memory consumption problem. Translation datasets could contain two to three million keys, although individual services typically accessed only a small subset. Cache refreshes introduced another issue: the service built a replacement cache in the background before atomically swapping it with the existing one. Both copies therefore occupied memory simultaneously, causing some services to run out of memory during refreshes.

Uber considered external caching and generating service-specific translation subsets, but both introduced additional operational or maintenance costs. The team instead adopted a hybrid storage design that preserves the existing Get() and Put() API. All keys remain available in memory for fast lookups, while translation values are stored in disk-backed files.

Uber's hybrid translation cache (Source: Uber Blog Post)

The implementation uses 256 shards, each with its own read-write mutex, in-memory entries, and disk file. Six-byte hash-based keys reduce metadata overhead, with a fallback to the original key format to handle collisions. A time-based global least recently used (LRU) policy manages in-memory entries. During cache refreshes, the system uses access history to preload translations accessed within the previous 15 minutes, avoiding a cold cache after the swap.

In one example, a translation dataset that previously consumed approximately 800 MB in memory required around 40 MB for metadata after the redesign. With a configured limit of 50,000 in-memory entries, actual memory consumption typically ranged between 80 MB and 120 MB. In another reported workload, approximately 30,000 of more than four million entries were loaded in memory, while the cache hit rate remained above 99.9%. The p99 latency for disk-backed cache misses stayed below 100 microseconds.

Memory usage decreased from approximately 2.7 GB to 750 MB (Source: Uber Blog Post)

Uber's experience highlights the cost implications of in-memory caching at scale. The team recommends minimizing object references to reduce GC overhead and measuring how frequently cached values are accessed. When only a small proportion of cached data is actively used, a hybrid approach can retain frequently accessed values in memory while moving the rest to disk.

Uber's hybrid translation cache (Source: Uber Blog Post )
Memory usage decreased from approximately 2.7 GB to 750 MB (Source: Uber Blog Post )