How do caches work across models? I would have thought that was very model speci... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

IanCal 43 days ago | parent | context | favorite | on: Nvidia Nemotron 3.5 Lightning and NeMo Switchyard

How do caches work across models? I would have thought that was very model specific - if not I’ve really misunderstood what’s getting cached.

armanckeser 43 days ago [–]

I am not sure the author of the comment you are replying to understands that LLM systems have prompt caches

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact