Paginate Entries using the Connection Specification

Instructor: [0:00] After some custom development, we want to make another change to our workshop. We plan to add a section to the detail page, previewing recommended adapters and Access files for each laptop.

[0:13] To do so, we can extend our sample query by a field recommended products and ask for the names and image URLs. In the result, we would receive a list of recommended products. For example, a USB to HDMI adapter, as well as a USB to SD card adapter.

[0:39] This only works well as long as the list never grows beyond the reasonable size to fetching one goal. In our case though, the list of Access files might include hundreds of different products.

[0:51] What to do in such a case? Pagination to the rescue. We add limit arguments to limit amount of recommended products, fetching them on request. With the page argument, we can fetch one specific page at a time.

[1:05] There are several issues, though, with this approach. One of them is that, we can't tell if there's a previous or next page.

[1:13] Before explaining all the other issues, I want to briefly show you the established best practice, solving all these issues at once, the relay cursor connections specification, or short, connections spec.

[1:27] It's a specification that defines a standardized way for exposing connections in GraphQL. While it originated in relay, it doesn't require us to use relay, but turned out to be a really good idea, and so became a best practice.

[1:44] Let me illustrate it by creating an example query to retrieve recommended products. We would use the argument, first, to limit the result and provide a cursor -- basically the ID of the last item of the previous page -- to the argument, after.

[2:02] If we want to fetch the previous page, we can provide last five and before, with a cursor.

[2:09] Now, instead of directly asking for the name and image, we get back edges and pageInfo. Edges return a list of nodes. In there, we can finally query for the name and the image. pageInfo contains hasNextPage, hasPreviousPage, startCursor, and then endCursor.

[2:34] While it's not obvious yet, let me tell you that this will solve all the problems with connections you can possibly imagine. Maybe not all of them, if you're very creative, but most of them.

[2:48] All right. Before we dive more into the explanations of how and why, let's talk about the elephant in the room. It looks ugly, and it's way more bloated than feels necessary.

[3:02] Nevertheless, in the next couple minutes, I hopefully will change your mind, and you'll consider using the connections spec a good choice.

[3:11] How do we get there? Let's take a step back and start from our initial pagination design and let one issue with a suitable solution after another.

[3:21] First, we need to retrieve the information if there's a previous and next page. To do so, we can add another field, recommend the product's page info, pull out the same arguments, and create a has next page, as well as has previous page. Not particularly exciting to duplicate the arguments.

[3:43] We can refactor it and nest both inside one field. Way better.

[3:54] All right, so next up. Our CLS team recommends us to show how often [inaudible] SSR has been brought to get out with the current product. Let's add to field bought to get a percentage to our query.

[4:08] At first sight, this looks like it's going to be completely fine. There's one issue though that will only show itself once we write down the type definitions. So let's do that.

[4:20] We add the field recommended products to product, and add the type recommended product connection. In there, we add the field products, returning a list of products. Then, we add bought to get a percentage to product.

[4:39] Exactly this is where we have a problem. Bought to get a percentage only makes sense in relation to another product.

[4:46] If you query for a single product, this field doesn't make sense at all. To tackle this issue, we can create a new type, recommended product with the field bought to get a percentage, and change our connection time.

[5:13] One thing that bothers me, though, is that there's a lot of duplication. This isn't necessary. Instead of spreading out all the fields, we can simply refer to the product.

[5:26] So far, so good. What I have to add here is the outcome. So far, is definitely more complex than referring to a list of product with the field recommended products in the connection type. On the other hand, it's a beautiful solution, allowing us to attach meet information.

[5:44] Now that our types are in a good state, we go back to our sample query and check how this type changes affect it. We change products to recommend products, and this name and image inside product.

[6:03] My next concern is the page argument. To explain why using a page index isn't a good idea, I best walk you through a couple of data fetching scenarios.

[6:13] We can start with a page index scenario that comes with [inaudible] issues. On the left side, we see a list of items stored in the data base. If decline request the first page with a limit of five, the server will return the first five items. Then, decline can ask for the second page and we'll receive the items 6 until 10. So far, so good.

[6:38] Let's start over. The client ask for the first page and receives the first five items. Before asking for the second page, something else is happening. Another user removes the items three and four from the list. This is obviously affecting the pages.

[6:59] Now, our client requests the second page and receives items 8 until 12. Our client is missing item six and seven. No warning, no indication that the client missed these entries. Depending on your context, this can be neglectable, but in some environments, missing a message while on entry, can lead to horrible outcomes.

[7:27] Can we prevent it? Of course. Cursors to the rescue. Not these cursors, but rather the concept of providing an ID as a reference for position in the list. How does this work? Let's go for the last scenario once again, but now with cursors.

[7:44] The client requests the first five items and leaving out the cursor, basically means giving the first page. Then, items three and four are removed. After that, the client requests the first five items after the cursor ID 5, because this is the last ID our client was aware of.

[8:04] As you can see, the item 6 to 10 are returned and no items are missing. Fantastic. That said, with our real time updates, users might still miss if entries have been deleted. Like in our case, item three and four are still on the list of the client.

[8:22] Nevertheless, at least the client won't miss a message, which is definitely better than missing them.

[8:28] What does this mean for our query? We replace "page" with "after" and passing the ID of the last item was called mCoursor. After convenience, we can add the field mCoursor to our page info.

[8:43] This allows us to fetch the next page, but how can we fetch the previous page using cursors? Simply by replacing the argument "after" with "before." In our implementation, we can make sure the once "before" is provided, we retrieve to previous items.

[9:00] Again, out of convenience, we can add the field "start cursor" to our page info.

[9:07] Now, we are at a point where it's really interesting to compare our current query with the connection spec example we wrote before.

[9:16] The structure is identical and the only difference is the naming of certain fields. This makes a lot of sense, because if we start to generalize these names and replace recommended products with edges, product with node and limit with last, we get the exact same result. Fascinating.

[9:39] By applying several common feature requests, we ended up with the connection spec.

[9:47] I hope with this little exercise, I could convince you that while the connection spec comes with some complexity, it's a really good idea. Make sure your scheme is ready for changing requirements. Pretty cool.

[10:01] Before we conclude the lesson, let us implement the type definitions so you have seen it at least once. We changed arguments to accept first integer, after ID, last integer, and before ID.

[10:16] Then we changed products to edges in the recommended product connection, and product to node in the recommended product. Since the recommended product is part of a connection, it also makes sense to post fix it with edge.

[10:32] At the final step, we add the type page info containing has next page, has previous page, start cursor, and end cursor. Then add it to our connection. We copy our example query and verify that it works.

[10:51] Great. To fetch the next page, we only need to replace the arguments and query for first five and after CDE. Works like a charm.