Instagram system design: a fan and a star, then one story to 184 friends

Follow a fan's follow, like and comment on Virat Kohli's post, then one story copied into 184 friends' trays, while a star's stories are fetched, never copied.

Follow, story, like, comment

  1. Rajat has found Virat Kohli on Instagram and taps his name. The gateway checks his login and his rate limit, and hands the request to Graph, which owns profiles and follows.
  2. Virat’s profile is read millions of times a day, so Graph doesn’t go to the database. It reads it from Memcache.
  3. Back on Rajat’s phone: 273 million followers. The photos come from the CDN, from a server near him.
  4. He taps Follow. Graph writes two rows in PostgreSQL: Virat in Rajat’s following, Rajat in Virat’s followers, each on its own user’s shard. Virat’s count goes up by one.
  5. The answer comes back the way it came, and Rajat’s button now says Following.
  6. Graph also puts a Followed event on the event stream, for Notify. It won’t buzz Virat for every new fan; it gathers the last few minutes of follows into one line.
  7. Rajat and 48 others started following you. His profile still reads 273 million: the exact count moved by one, but the rounded number doesn’t.
  8. Now the story. Rajat taps the ring around Virat’s photo, and the request goes to Story.
  9. Virat’s stories sit in one list, read from Memcache. It’s never copied to his 273 million followers; each of them asks for this same list.
  10. The video streams from the CDN. Once it’s watched, the ring turns grey, and his app tells Story he has seen it.
  11. Story adds him to Virat’s viewers, in Cassandra, a database built for huge numbers of small writes. Virat isn’t told: a view is a row, not a notification.
  12. He opens Virat’s latest post. Its caption and likes came with the profile, so only the photo is fetched, from the CDN.
  13. He double-taps. The heart turns red at once, because the app doesn’t wait, and the like goes to the Like service.
  14. It writes one row: this post, this person. Tap again and that row is already there, so a like can never count twice.
  15. Thousands of likes a minute would queue on a single number, so the count is split across many small counters, added up when it’s read.
  16. Notify groups these too. Virat’s phone: Rajat and 12,940 others liked your post.
  17. Last, a comment: Take a bow, King. Before it’s saved, Safety checks it for spam and abuse, and for words Virat has chosen to hide.
  18. It’s clean, so Comment saves it, and the answer comes back the way it came.
  19. The comment appears under the post on Rajat’s phone.
  20. And one more grouped line on Virat’s phone: Rajat and 1,830 others commented.
  21. Four actions. Virat’s phone changed three times, each with many fans in one line. A celebrity is read from cache, counted in pieces, and told in groups.

Post a story

  1. RCB have won, and Rajat is putting it on his story. Watch it reach the five friends on the right.
  2. First the photo goes up to Media in small chunks, so a weak signal resends only the chunk it lost.
  3. Media keeps the original in object storage, and makes the smaller sizes that phones will download.
  4. The app gets an upload id back. The photo is stored, but it isn’t a story yet, so the app asks Story to post it.
  5. Before anyone can see it, Safety checks the photo.
  6. Story saves it with a twenty-four-hour expiry: after that it leaves every tray, and only Rajat can see it, in his archive.
  7. The answer comes back, and a ring appears around Rajat’s photo.
  8. Story also puts a Story Posted event on the stream, and a fan-out worker picks it up.
  9. It asks Graph who follows Rajat: 184 people.
  10. It writes the story into each of their trays, the row of stories at the top of the app: 184 small writes, done in moments.
  11. DK opens Instagram. No notification came; his app simply asks Story for his tray.
  12. It was filled the moment Rajat posted, so it’s one quick read, and Rajat’s ring comes first.
  13. He taps it. The photo comes from the CDN, his app tells Story he has seen it, and DK becomes Rajat’s first viewer.
  14. Faf opens the app next. His tray was ready too, and the photo comes from the same CDN copy: Rajat uploaded it once, and the edge serves it to everyone.
  15. Similarly, ABD, Chris and Virat watch it too. Each view is one more small write to Rajat’s viewers list, and none is pushed to his phone: a story with a million views is a million rows.
  16. Later, Rajat opens his own story to see who has watched, and his app asks Story for the viewers.
  17. Five rows come back, and his story shows five viewers: DK, Faf, ABD, Chris and Virat.
  18. One photo, stored once, copied into 184 trays in moments, and seen by five friends. For Virat’s 273 million followers, that copying would take hours, so his stories are fetched instead.