Starting today, Xata is out of private Beta! 

Goodbye waiting list 👋, you can go ahead and [create an account](https://app.xata.io/signup). If you like to learn by doing and want to dive in right now, we promise you’re going to have a production-ready database, including high availability, with a data model, and test data in it in a matter of minutes. [Try it](https://app.xata.io/signup)!

After more than a year of work and receiving and addressing feedback from over 700 developers, we are very excited to show you what we’ve been up to. 

So let us take you on a quick tour.

## Why Xata?

Our mission is to take the hard data problems, solve them using the industry's best practices, and then offer them to you in the most easy-to-use way possible. 

The hard data problems include running PostgreSQL at scale and with high availability, replicating the data into Elasticsearch for search and analytics, edge caching, major-version upgrades, data migrations, observability, and so on. For more details around how we abstract away this complexity, see the How it Works documentation page.

![Xata technical diagram](/images/docs/xata-cell.png)

We call the type of service we’re building at Xata a [Serverless Data Platform](https://xata.io/docs/intro/serverless-data-platform), and we think it’s the next logical evolution after Database as a Service (DBaaS).  Similar to how platforms like Vercel, Netlify, or Cloudflare Pages leverage CDNs, serverless and edge functions, to host your app globally and give you a simple developer experience, Xata does the same for the data infrastructure.

When we say that we offer them to you in the most easy-to-use way possible, we mean things like:

- as serverless as it gets (see our blog post on [serverless databases](https://xata.io/blog/what-is-a-serverless-database))
- a single consistent API across multiple types of data stores (relational DB, search/analytics engine, in the future caching and queues)
- a web UI that makes working with the data easy during and after development
- ORM-like high-level clients for the most popular programming languages

## Show me the Code

Xata has a relational data model, with a strict schema (schemaful) and support for JSON-like objects. Records are grouped into tables, which are grouped into databases. Xata supports rich column types and relations between tables can be represented via link columns, which are similar to foreign keys.

A record in Xata can look something like this:

```json
{
  "name": "John Doe",
  "email": "john@example.com",
  "age": 42,
  "address": {
    "street": "123 Main St",
    "city": "New York"
  },
  "labels": ["admin", "user"],
	"team": "rec_cdcp6a98ru583kilecog"
}
```

The above shows a column of type object (`address`) with two sub-keys (`street` and `city`). It also shows a column of type email and an array-like column (`labels` of type `multiple`).

![Schema editor](/images/blog/launch/schema-editor.mp4)

The `team` column is a **link column**, pointing to a record in another table, similar to a foreign key. Link columns make it easy to represent relationships, see more in our [Data model](https://xata.io/docs/concepts/data-model) docs.

With the TypeScript/Javascript SDK, you can query the above record with something like this:

```tsx
const users = await xata.db.Users.select(["*", "team.*"])
    .filter({ "address.city": "New York" })
    .getMany();
```

In the above, because `team.*` was explicitly requested, the results will return all the columns from the linked `Teams` table as well. Behind the scene, Xata is doing a JOIN, but we expose it in a simpler way in our APIs.

You know what’s the best part about the above? If you’re using TypeScript, both the parameters and the results are fully typed. So you can easily use auto-complete to get column names, for example, and the types are going to catch mismatches between the code and the database schema. Type safety with no effort!

You can see a lot more examples of how to use both the API and the TypeScript SDK in our [API Guide](https://xata.io/docs/api-guide/get).

*Psst, there’s an easier way!*

You **can** read the API guide to see all the query options, but the truth is you don’t really have to 😀. Instead, you can configure your query in the UI and then hit the “Get code snippet” button. It will give you the exact query that was executed to get the results that you see on the screen. We believe that our code snippets are not only the fastest way to develop on top of Xata but also the best way to learn Xata.

![Get Code Snippet](/images/blog/launch/get-code-snippet.mp4)

## Branches and Zero-Downtime Migrations

A database in Xata can have multiple branches, which are first-class citizens in the API. Branches are full logical databases that might or might not exist on the same set of physical instances. When you start a new branch from an existing one, the schema is copied, but the data is not automatically available in the new branch. As you create a development branch from a production branch, you can copy and sanitize parts of the data or initialize the branch with test data.

For each branch, we record its schema history in a list of “commits”. These commits contain operations like “add a column”, “rename a column”, “add a new table”, “delete a table”, and so on. We also record which branch is created from which. This history information allows us to create the correct migration diffs and perform the minimum set of operations required for a migration.

There are multiple benefits to having database branches, which is why they are being adopted by multiple database products recently. They:

- enable development workflows that match git-workflows, for example having a branch for development, one for staging, one for production, etc.
- enable deployment previews on platforms like Vercel and Netlify.
- enable standardized workflows for applying schema migrations safely.

The last one is particularly interesting because migrating database schemas is usually easy with small amounts of data, but becomes very painful when dealing with large amounts of data. In the NoSQL wave of DB products, this issue was avoided by going schemaless. However, not having a schema introduces its own class of problems, so lately the pendulum is swinging back toward databases with strict schemas. 

Because we have control over the data model and because Xata has native concepts of migrations and migration requests, we can ensure that all supported migration types are safe and they don’t lock the database. That’s why we call our migrations **zero-downtime** migrations. They constrain you in what types of migrations exist, but you can have the confidence that your data model will keep working at scale.

## Free-Text Search

As we have shown earlier, we store the data in PostgreSQL, but we also replicate it to Elasticsearch. This enables us to offer significantly more advanced free-text search compared to the free-text-search support from PostgreSQL. This includes:

- [fuzinness control](https://xata.io/docs/api-guide/search#fuzziness-and-typo-tolerance) so that you can have typo tolerance in your search.
- [column weights](https://xata.io/docs/api-guide/search#column-weights), so you can value matches in some columns more. For example, you can boost matches in the “title” column more than matches in the rest of the columns.
- [numeric boosters](https://xata.io/docs/api-guide/search#numeric-booster), so you can boost records by a numeric field, for example by “rating”.
- [date boosters](https://xata.io/docs/api-guide/search#date-booster), so you can boost records that are more recent in time.
- [value boosters](https://xata.io/docs/api-guide/search#exact-value-booster), so you can boost records with a particular column value, for example, by “category”.

This allows you to build the perfect search experience for your users, without having to use a different service or product and worrying about how to replicate the data. 

Since relevancy tuning can be tricky business, we provide a UI so you can do it visually, see the results of it in real-time, then hit the Get Code Snippet to get the code that you need in your app.

![Search](/images/blog/launch/search.mp4)

### Summaries and Aggregations

Xata offers two ways to aggregate your data: summaries, which are executed in PostgreSQL, and aggregations, which are executed in Elasticsearch.

An example of a relatively complex visualization that can be created with the [Aggregation API](https://xata.io/docs/api-guide/aggregate) could be: a multi-line chart, where each line represents a movie genre, and the Y axis represents the average rating of the movies in that genre, per year. This chart can be obtained with a single aggregation request, looking something like this:

```tsx
const results = await xata.db.titles.aggregate({
  "movieGenres": {
    topValues: {
      column: "genre",
      size: 50
      aggs: {
        "byReleaseDate": {
          dateHistogram: {
            column: "releaseDate",
            calendarInterval: "year",
          },
          aggs: {
            "avgRating": {
              average: {
                column: "rating"
              }
            }
          }
        }
      }
    }
  },
})
```

You can find more examples in the [API guide](https://xata.io/docs/api-guide/aggregate). We’d don’t yet have a UI to help you build this type of aggregations, but if you know us by now, we’re going to have one soon.

### And we’re just getting started

With today’s public launch, it is day 0 at Xata. Our mission to simplify the way developers work with data gives us a strong focus on what features to add to the product. We believe that Xata, as a [serverless data platform](https://xata.io/docs/intro/serverless-data-platform), is the perfect companion for full-stack frameworks like Next.js, Remix.run, SvelteKit and others.

We’re very excited about upcoming features like Edge Functions and Caching (currently in private Beta, see more here),  integrations into Vercel and Netlify, global data replication, and many others. [Stay tuned](https://twitter.com/xata), there’s so much more to come.

### How free is free?

Many DBaaS and BaaS vendors offer a free-tier plan, but usually, it doesn’t include redundancy or high availability, and it’s often “for development” only with limitations based on activity. 

Our [free tier](https://xata.io/pricing) is explicitly recommended for production. We want you to build your personal projects, non-profit web apps, and fun apps on top of Xata, and you shouldn’t have to pay for it. You can also freely collaborate with others and be part of multiple teams (up to 40 workspaces per user). The limitations of the service are clearly stated on the [pricing page](https://xata.io/pricing) and on the [limits](https://xata.io/docs/concepts/limits) page in the docs.

If you are a seed-stage startup, a non-profit, or you were in our Beta program, we can increase the limits further, [contact us](mailto:info@xata.io) and we’d be happy to discuss. You can think of us as your team of DevOps people experts in databases.

More than that, we will be on [discord](https://xata.io/discord) to help you along the way. Make sure to say hi so we can send you some swag when you publish your work.

If you read this far but didn’t try Xata yet, you should definitely [do it now](https://app.xata.io/signup)! 🦋

Starting today, Xata is out of private Beta! After more than a year of work, receiving and addressing feedback from over 700 developers, and stretching our designers thin in the past weeks, we are very excited to show you what we’ve been up to. 

Xata Is Now Live! 🚀

A post detailing how an app was built on Xata with Next.js 13 using its powerful features.

Searching for Retro Games With Xata & Next.Js 13

Next.js 13 is the latest version of the popular full-stack framework based on React. In this post, we'll take a look at the new data fetching features and how Xata can be seamlessly used with their *next* big version.

As announced at the Next.js Conf, Next.js 13 offers a completely new solution for file-system routing, nested layouts, incrementally streamed rendering, and component-level data fetching with React Server Components (RSC). You can find out more in [the Next.js blog announcement](https://nextjs.org/blog/next-13) or replay the keynote from Next.js Conf 2022.

At Xata, we are very excited about this new data fetching mechanism. It removes even more barriers between server and client code and is built on top of web standards, like the fetch API, and it works out of the box with our TypeScript SDK.

### React Server Components and Suspense for Data Fetching

Prior to Next.js 13, data fetching on Next.js was usually done with `getStaticProps` and `getServerSideProps`, two powerful functions that allowed per page define server-side code that would be serialised and sent over the network to the user’s react code in the browser.

React 18 and Next.js 13 are offering a new way to build and run React Components either on the client or the server. Paired with Suspense for Data Fetching, now your application can safely access backend resources or fetch data without leaking sensitive data.

Moreover, taking advantage of the new nested layout mechanism allows for instant rendering of page parts that do not specifically require data. Your application can show a loading state for parts of the page that are fetching data and reduce the time to load the content.

### How the Xata SDK can be used with Next.js 13

At Xata we are building a serverless data platform that is an excellent companion for Next.js. 

Since the Xata SDK is built on top of the same standards as the new data fetching mechanism from Next.js 13, you can already use them together and it will work out of the box. Let’s see an example!

```tsx
const xata = getXataClient();

export default async function Blog() {
  const posts = await xata.db.posts.getAll();

  return (
    <div>
      <h1>Blog posts</h1>

      {data.map((post) => (<h2>{post.title}</h2>))}
    </div>
  );
}
```

Simple, right? As you can see, by only defining the Xata SDK client outside of the server component, you can run any async query or function and the results will be available to be used in the component.

In a future update, we will expose the new cache strategy and data revalidation properties, so that when using Xata and Next.js you can take full advantage of static site generation, server-side rendering, and incremental static regeneration with the new `use` hook.

Stay tuned for our documentation updates that will showcase the newest way of fetching and rendering data with Next.js.

### Conclusion

We are very proud of our partnership with Vercel, sharing a common vision on how modern and performant web applications can be built.

If you want to discuss more about the newest features in Next.js or you have already tried the new data fetching mechanism come to our [discord](http://xata.io/discord) and chat with us!

Next.js 13 is the latest version of the popular full-stack framework based on React. In this post, we'll take a look at the new data fetching features and how Xata can be seamlessly used with their next big version.

The Future of Data Fetching With Next.Js 13

With a database access key being sent out from the browser, your database is readable and writable by anyone. They may read the key and therefore use the key to create malicious connections to your database. These actions (attacks) are called **exploits**, and the data retrieved from such exploits are called **leaks**. Yes, like the famous “data leaks” we hear about in the news. Whether your leaked app makes to the news or not will depend on how big and influential your user-base is, but the problem is still the same. Someone with the key can even delete all of your data if they so choose.

When building an app, a common pattern is to build an API (Application Program Interface) that will work as a bridge between your database and the client-side app. Said API will expose a certain number of endpoints depending on how it is designed. For example:

![Diagram of App-API-DB architecture](https://i.imgur.com/NgpZvIp.jpg)

This architecture is not only common, it's a requirement. It happens because a database requires **access keys**. As anything valuable, user data must be stored behind locked doors. Wherever the query originates from, must add these **access keys** (often referred to as **access tokens** as well) to the request. This is why any communication with a database must happen in a **private connection**. 

A browser is a **public** interface. Whatever data and code an app has in the browser is **exposed the public**. Now connect the dots: if everything on the browser is available to the public, and a query (request) goes from the browser straight to the database, what happens to the secure key it must carry to actually retrieve the data? You guessed it: the key is readable, as said in security contexts, it is **exposed**. 

## Querying Xata

To make things more ergonomic, Xata has a [Software Development Kit (SDK)](https://xata.io/docs/sdk/getting-started) we call [client-ts](https://github.com/xataio/client-ts). Under the hood, there are many best practices we enforce. It also generates TypeScript types for your data request and responses based on your schema - that is the Object Relational Mapping (ORM) side of our SDK. 

But arguably, the most important aspect of our SDK is that it will securely query Xata using your environment variables (`process.env`) the `XATA_API_KEY`, which is the key required to query your database. There multiple different ways to expose this environment variable to your running app, for example:

- when calling the script:

```bash
XATA_API_KEY=my_api_key node ./script/query.js
```

- more commonly, via the `.env` configuration file:

```bash
XATA_API_KEY=my_api_key
```

When deploying your app, you must find out how to define Environment Variables to the deployment script. Each PaaS (Platform as a Service) has a different way of doing that. Have a look for examples at [CircleCI](https://circleci.com/docs/env-vars/), [Cloudflare Workers](https://developers.cloudflare.com/workers/platform/environment-variables/), [Netlify](https://docs.netlify.com/configure-builds/environment-variables/), or [Vercel](https://vercel.com/docs/concepts/projects/environment-variables).

The `process.env` namespace is not available in browser environments so if using the SDK, you will experience *friction* when attempting to query your Xata database from the browser. **This is intentional and by design.** Essentially, `process.env.XATA_API_KEY` will be `undefined`,  and the SDK will not be able to fire your request (thus preventing you from leaking access to your database), and you will see a runtime error like:

```bash
error Error: Option apiKey is required
    at XataClient.parseOptions_fn (index.mjs?4597:3090:1)
    at new _a (index.mjs?4597:3053:1)
    at XataClient._createSuperInternal (_create_super.mjs?05d7:12:1)
    at new XataClient (xata.codegen.ts?b4b0:32:24)
    at getXataClient (xata.codegen.ts?b4b0:42:14)
    at eval (stepone.js?34b1:101:47)
```

If on a Node.js environment, this means the SDK cannot read your API key. You can solve this by using one of the methods mentioned above to expose your access key to your runtime. If using the `.env` approach, you may need a package like [dotenv](https://www.npmjs.com/package/dotenv) to read those values.

If on a browser environment, this error **********************************************should not be “fixed”**********************************************. As explained in the first section of this article: exposing your Xata API keys to a client-side runtime like a browser is a huge security issue. For more information, read our documentation about [Securely Talking to Xata](https://xata.io/docs/quickstart/updating-data-with-xata#securely-talking-to-xata). 

## Creating Apps and Connecting to Xata

The path of least resistance to create an app and connect it to Xata is to use a full-stack framework such as [Next.js](https://nextjs.org/docs), [Remix](https://remix.run/docs/en/v1), [Nuxt](https://nuxtjs.org), [Gatsby](https://www.gatsbyjs.com/docs/), and others. Such apps offer a way of creating **serverless functions** within them. In Next.js for example, any files in `./pages/api` contain these functions. But what are they? In a nutshell serverless functions are pieces of code that will run only on the server, allowing developers to query their database there and thus remaining close to the app logic and more importantly, ***out of the unsafe browser environment.***

When creating a pure client-side app ([Astro](http://astro.build), [Eleventy](https://www.11ty.dev/), [Qwik](https://qwik.builder.io/), [React](https://beta.reactjs.org/), [Solid](https://www.solidjs.com/), [Svelte](https://svelte.dev/), [Vue](https://vuejs.org/), etc.), the database request will not be able to originate from the app. Instead, refer to the initial architecture we outlined in this post: first hit an API, then the API connects to the Database. This keeps your API keys secret. 

### What are Serverless Functions?

Different than a long-standing server that will continue to run even when there are no requests coming in or out, a serverless function will cease activity and go into “sleep” whenever its execution is finished. This makes serverless functions a perfect use-case to bridge connections between your app and your database.

Additionally, serverless providers like [AWS Amplify](https://aws.amazon.com/amplify/), [Cloudflare Workers](http://workers.cloudflare.com), [Netlify](http://netlify.com), and [Vercel](http://vercel.com) will provide developers with a no-obstacle deployment. You define your function handler (most receive 2 parameters: `request` and `response`). 

## Connecting a Serverless Function to Xata

Creating a serverless function is basically as straightforward as creating a JavaScript file.

```jsx
// In Next.js, this would be ./pages/api/get-data.ts

export default function handler (request, response) {
  // fetch stuff from Xata
  const data = await xata.db.myDb.getMany();
  response.end(data.records);
}
```

To understand how to create one, and how to design your API, we have a [Vercel Serverless Function Sample](https://github.com/xataio/examples/tree/main/apps/starter-vercel-serverless-functions) in our example repository. Do not forget to create and add your API Keys to the deployment routines within your dashboard, otherwise the Xata Client will not have a key to fire requests with.

For a more involved example, refer to the [Quickstart in our documentation](https://xata.io/docs/quickstart/updating-data-with-xata).

## Conclusion

We hope this has made it clearer how to interact with your Xata database from within and outside your apps. Please feel free to create an issue in our [examples repository](https://github.com/xataio/examples) if there is a specific use-case that you would like us to prioritize in showing how to address. 

Additionally, we cannot stress it enough that **we love open-source** and there is some swag waiting for everyone who ships a community sample app, or creates a PR to contribute to any of our projects (examples included!).

Creating a secure connection between an application and a Xata database. How to fetch data from pure client-side apps without exposing your security keys.

Securely Querying Your Database on Xata (and Why Browser-Based Requests Are Not Secure)

At Xata, we are currently in closed beta. To support this, we want to have the best documentation we can. This, of course, is easier said than done, so we are experimenting by moving things around, listening to user feedback, and so on.

Our documentation is written in [Markdown](https://www.notion.so/A-Journey-to-Bulletproof-Links-in-Documentation-f108c0ed0acd464bb1e6310519d19e37), served by [Next.js](https://www.notion.so/A-Journey-to-Bulletproof-Links-in-Documentation-f108c0ed0acd464bb1e6310519d19e37). This has a few advantages:

- we have total freedom in the design,
- we can fetch Markdown documents from GitHub to keep the documentation of a given product close to its code,
- we can have custom components: graphs, Twitter cards, etc.

So far, we had no way to make sure that links between pages were working and that they didn’t just 404. After moving things around, even after checking and rechecking everything manually… **we had 28 broken links in our production documentation.**

![This is fine meme](/images/blog/bulletproof-docs-links/Untitled.png)

This is fine, we all make mistakes, but let’s learn from this together!

## The Plan

The first step is to list our requirements so we have our goal well defined. We want quick feedback if something gets broken, so we can fix it before merging to production. For that, we need to be able to run the docs consistently in different environments:
   - local
   - preview/ staging
   - production

With this in mind, one constraint immediately becomes clear to us: this would be way easier if we don’t have a link checker that needs to run on a server.

If we want to avoid a server, we need to extract the information:

- the internal links in my markdown files
- the directory structure.

## The Proof of Concept

To achieve our final goal, we need a reliable way to extract links from our existing documentation and see if the link *actually goes somewhere*. This is a lot of files, with a lot of links. To simplify our problem, let’s omit the real markdown documents for now and focus on the link extraction part.

> Our docs are at https://xata.io/docs by the way, if you’d like to explore them and use Xata yourself.
> 

1. First, we need to read, parse, and understand a Markdown document. Our parser of choice is  `unified`. It returns an **Abstract Syntax Tree (AST)**. Let’s give it a Markdown document and see what happens.

```tsx
import { describe, expect, it } from "vitest";

import { unified } from "unified";
import remarkParse from "remark-parse";
import remarkStringify from "remark-stringify";

describe("markdown link checker", () => {
  it("should extract all links from a given document", async () => {
    const document = `
# Title
			
this is a paragraph with [a link](/rest-api/get)

## Another title

and another [link](/rest-api/delete)
`;

    await unified()
      .use(remarkParse)
      .use(() => {
        return function transform(tree) {
          console.log(tree);
        };
      })
      .use(remarkStringify)
      .process(document);

    expect(true).toEqual(false);
  });
});
```

When we `console.log` the output, we get this:

![Untitled](/images/blog/bulletproof-docs-links/Untitled%201.png)

And just like this, we can see our markdown content in a structured way, cool no? (we can also use [https://astexplorer.net/](https://astexplorer.net/) to see explore our AST)

2. Extract the links

To do this, we can use `unist-util-visit`, which gives us a very convenient and type-safe way to “visit” our AST.

What does “visit our AST” mean? Every time we have a node with  `type === "link"`, even very deep inside `children`, a function is called with this node as argument! This function is called a **visitor function**. Good news, this is exactly what we want! 😀

Instead of using our previous raw  `tree` object and iterating through every node looking for those `link` objects, we can write this:

```tsx
import { describe, expect, it } from "vitest";

import { unified } from "unified";
import { visit } from "unist-util-visit";
import remarkParse from "remark-parse";
import remarkStringify from "remark-stringify";

describe("markdown link checker", () => {
  it("should extract all links from a given document", async () => {
    const document = `
# Title
			
this is a paragraph with [a link](/rest-api/get)

## Another title

and another [link](/rest-api/delete)
`;

    const links: string[] = [];

    await unified()
      .use(remarkParse)
      .use(() => {
        return function transform(tree) {
					// Let's extract every link!
          visit(tree, "link", (linkNode) => {
            links.push(linkNode.url);
          });
        };
      })
      .use(remarkStringify)
      .process(document);

    expect(links).toEqual(["/rest-api/get", "/rest-api/delete"]);
  });
});
```

That’s essentially all we need for a proof of concept (POC), we are now more confident that we should be able to extract every link from our documentation. Let’s try with some real data!

## The Real Deal

First, we need a way to load all our Markdown files. Luckily, [npm](https://www.notion.so/A-Journey-to-Bulletproof-Links-in-Documentation-f108c0ed0acd464bb1e6310519d19e37) already has a solution for us. 😀

```tsx
import { describe, expect, it } from "vitest";

import { unified } from "unified";
import { visit } from "unist-util-visit";
import remarkParse from "remark-parse";
import remarkStringify from "remark-stringify";
import { getAllFiles } from "get-all-files";
import { readFile } from "fs/promises";

describe("markdown link checker", async () => {
  for await (const filename of getAllFiles("./content")) {
    if (!filename.endsWith(".md")) continue;
    it(filename, async () => {
      const document = await readFile(filename, "utf-8");
      const links: string[] = [];

      await unified()
        .use(remarkParse)
        .use(() => {
          return function transform(tree) {
            visit(tree, "link", (linkNode) => {
              links.push(linkNode.url);
            });
          };
        })
        .use(remarkStringify)
        .process(document);

      expect(links).toEqual([]);
    });
  }
});
```

We are iterating on every file from the `/content` directory and reusing our previous logic, replacing the `document` with our actual markdown content!

And… it works! Not useful yet, but we are going in the right direction!

![Untitled](/images/blog/bulletproof-docs-links/Untitled%202.png)

This is also a nice opportunity for us to see what links we actually have! And we can already spot some patterns that we need to deal with as external links (example: [`https://stackoverflow.com/questions/4423061/how-can-i-view-http-headers-in-google-chrome`](https://stackoverflow.com/questions/4423061/how-can-i-view-http-headers-in-google-chrome)) and links with anchors (example: `/cli/getting-started#code-generation`)

First, let’s exclude external links, they are out of our scope since we’re not interested in *other people’s 404s.* We may add this later since we don’t want to link to broken content in the longer term, but for now **scope hammering keeps us focused**.

```tsx
if (linkNode.url.startsWith("/")) {
  links.push(linkNode.url);
}
```

Let’s improve the output of our test: we want one `describe` per Markdown document, and one `it` per link. This is our target output (for now):

![Untitled](/images/blog/bulletproof-docs-links/Untitled%203.png)

So we have:

```tsx
describe(filename, async () => {
	// ...
	links.forEach((link) => {
        it(`should have "${link}" defined`, () => {
          expect("todo").toBe("done");
        });
      });
	})
})
```

And to finish this part, we need to check if the file exists, `fs.existsSync` will do the job:

```tsx
import { describe, expect, it } from "vitest";

import { unified } from "unified";
import { visit } from "unist-util-visit";
import remarkParse from "remark-parse";
import remarkStringify from "remark-stringify";
import { getAllFiles } from "get-all-files";
import { readFile } from "fs/promises";
import { existsSync } from "fs";

describe("markdown link checker", async () => {
  for await (const filename of getAllFiles("./content")) {
    if (!filename.endsWith(".md")) continue;
    describe(filename, async () => {
      const document = await readFile(filename, "utf-8");
      const links: string[] = [];

      await unified()
        .use(remarkParse)
        .use(() => {
          return function transform(tree) {
            visit(tree, "link", (linkNode) => {
              if (linkNode.url.startsWith("/")) {
                links.push(linkNode.url);
              }
            });
          };
        })
        .use(remarkStringify)
        .process(document);

      links.forEach((link) => {
        it(`should have "${link}" define`, () => {
          expect(existsSync(`./content${link}.md`)).toBeTruthy();
        });
      });
    });
  }
});
```

### Dealing with Anchors

Now, this is where, personally, I started by removing the anchor and called this done.

```tsx
it(`should have "${link}" define`, () => {
  const [path] = link.split("#");
  expect(existsSync(`./content${path}.md`)).toBeTruthy();
});
```

After a coffee and some time to think, I realized—we can do better! And also, I’m sure we have some broken anchors. (Spoiler alert, I found one 😅)

The first problem is that we need to have a dictionary of all anchors per document. So far, our AST logic is in middle of my unit test. Time to refactor!

Let’s extract our current logic into a function:

```tsx
/**
 * Retrieve useful information from a markdown file.
 *
 * @param path file path
 */
async function parseMarkdown(path: string) {
  const document = await readFile(path, "utf-8");
  const links = new Set<string>();

  await unified()
    .use(remarkParse)
    .use(() => {
      return function transform(tree) {
        visit(tree, "link", (linkNode) => {
          if (linkNode.url.startsWith("/")) {
            links.add(linkNode.url);
          }
        });
      };
    })
    .use(remarkStringify)
    .process(document);

  return {
    links,
  };
}
```

Let’s analyze what this method is doing in plain English:

- We’re taking a `path: string` as input
- We’re reading the content of the file that has this `path`
- We’re instatiating a `links` set, ready to be filled
- We have our AST visitor that:
    - on each node that fulfills the condition `type === "link"`
    - if the url starts with `/` (so it’s an internal link), we add it to `links`
- We return the consolidated `links` in an object

Now, we can extract the `anchors` following the same pattern

```diff
async function parseMarkdown(path: string) {
   const document = await readFile(path, "utf-8");
   const links = new Set<string>();
+  const anchors = new Set<string>();

   await unified()
     .use(remarkParse)
		 .use(() => {
		     return function transform(tree) {
		       visit(tree, "link", (linkNode) => {
		          if (linkNode.url.startsWith("/")) {
		            links.add(linkNode.url);
		          }
		        });
         });
+        visit(tree, "heading", (headingNode) => {
+          if (headingNode.children[0].type === "text") {
+            anchors.add(kebab(headingNode.children[0].value));
+          }
+        });
       };
     })
     .use(remarkStringify)
     .process(document);

   return {
+    anchors,
     links,
   };
 }
```

In Markdown, an anchor is any title converted to kebab case. For example, if I have a title `## Create Table` in `/getting-started`, I can point to `/getting-started#create-table`

Time to try our brand new function! Remember, we are just doing small steps—no need to run.

```tsx
import slash from "slash";

type FilePath = string;

describe("markdown link checker", async () => {
  // Collect all the data
  const pages = new Map<
    FilePath,
    { anchors: Set<string>; links: Set<string> }
  >();

  for await (const filename of getAllFiles("./content")) {
    if (!filename.endsWith(".md")) continue;

    pages.set(
      slash(filename)
        .replace(/^\.\/content/, "") // remove `/content`
        .slice(0, -3), // remove `.md`
      await parseMarkdown(filename)
    );
  }

  console.log(pages);
});
```

This test yields this output:

![Untitled](/images/blog/bulletproof-docs-links/Untitled%204.png)

Again, since we’re not worried about the entire problem, we can take our time to clean and prepare our object for the next step:

- Use `slash` so we don’t have those nasty backslashes (windows… my old friend…)
- Remove the `/content` and `.md` from my `path` so we have the same pattern as `anchor`

We can also spot a little problem here, did you see it? The `sidebar-position-5\nsidebar-label-api-keys` entry is not a heading! It’s a `yaml` node and needs to be parsed with the `remarkFrontmatter` plugin!

The source:

```tsx
---
sidebar_position: 5
sidebar_label: API Keys
---
# API Keys
```

We need to add `remarkFrontmatter` to the stack:

```tsx
import remarkFrontmatter from "remark-frontmatter";

// ...
await unified()
    .use(remarkParse)
    .use(remarkFrontmatter) // <- Just here
    .use(() => { /* ... */ } )
```

## The Wrap Up

This is the final version (the one used in our actual documentation repository!)

```tsx
import { describe, expect, it } from "vitest";
import { readFile } from "fs/promises";
import { unified } from "unified";
import { visit } from "unist-util-visit";
import remarkParse from "remark-parse";
import remarkStringify from "remark-stringify";
import remarkFrontmatter from "remark-frontmatter";
import { getAllFiles } from "get-all-files";
import { kebab } from "case";
import slash from "slash";

type FilePath = string;

describe("markdown link checker", async () => {
  // 1. Collect data from `/content`
  const pages = new Map<
    FilePath,
    { anchors: Set<string>; links: Set<string>; isRef: boolean }
  >();

  for await (const filename of getAllFiles("./content")) {
    if (!filename.endsWith(".md")) continue;

    pages.set(
      slash(filename)
        .replace(/^\.\/content/, "") // remove `/content`
        .slice(0, -3), // remove `.md`
      await parseMarkdown(filename)
    );
  }

  // 2. Generate unit tests
  Array.from(pages.entries()).forEach(([page, def]) => {
    if (def.links.size === 0) return;
    describe(page, async () => {
      Array.from(def.links.values()).forEach((link) => {
        it(`should have ${link} define`, async () => {
          const [path, anchor] = link.split("#");
          expect(pages.has(path)).toBeTruthy();
          if (anchor && !pages.get(path).isRef) {
            expect(pages.get(path).anchors.has(anchor)).toBeTruthy();
          }
        });
      });
    });
  });
});

/**
 * Retrieve useful information from a markdown file.
 *
 * @param path file path
 */
async function parseMarkdown(path: string) {
  const document = await readFile(path, "utf-8");
  const links = new Set<string>();
  const anchors = new Set<string>();
  let isRef = false; // `true` if the markdown content is `See …`

  await unified()
    .use(remarkParse)
    .use(remarkFrontmatter)
    .use(() => {
      return function transform(tree) {
        visit(tree, "link", (linkNode) => {
          if (
            linkNode.url.startsWith("/") &&
            !linkNode.url.startsWith("/api-reference")
          ) {
            links.add(linkNode.url);
          }
        });

        visit(tree, "heading", (headingNode) => {
          if (headingNode.children[0].type === "text") {
            anchors.add(kebab(headingNode.children[0].value));
          }
        });

        visit(tree, "paragraph", (paragraphNode) => {
          if (
            paragraphNode.children.length === 1 &&
            paragraphNode.children[0].type === "text" &&
            paragraphNode.children[0].value.startsWith("See ")
          ) {
            isRef = true;
          }
        });
      };
    })
    .use(remarkStringify)
    .process(document);

  return {
    anchors,
    links,
    isRef,
  };
}
```

Few highlights:

- We don’t need `fs.existsSync` anymore, we already have everything we need!
- We ignore `/api-reference` links, they are generated from our an external document at build time, so it’s out of our current scope.
- We added a last concept of `isRef`. This is again a very specific edge case that I don’t want to deal with, we have a part of our documentation fetched from GitHub with the following pattern:
    
    ```markdown
    See [link]
    ```
    

During this journey, I spotted 1 broken link and 1 broken anchor, but more importantly, I’m way more confident that we will not ship broken links in our documentation anymore. 

**This is what’s most important to us as developers: confidence in our code.** Through these tests and this approach, we have a little more of that with our documentation.

## The Conclusion

I hope you learned as much as I did! Please borrow ideas & code from this article and iterate from this. Unit tests are sometimes seen as something that slows us down or a boring thing to do, but I personally had a lot of fun playing with `unified` and Markdown’s AST, and I’m 100% convinced that this was not a waste of time—you can’t imagine how much time I spent on the documentation clicking every link, *and I still missed 2 of them!* This 100-line implementation is way more efficient than my manual QA 😅.

Happy hacking!

How we permanently fixed the problem of broken internal links in our documentation.

A Journey to Bulletproof Links in Documentation

At Xata we strive to be the **serverless database with the best developer experience** (DX) in the world. This is an ambitious goal and all of us are fully aware of that, but we have our vision and I can say confidently we are putting in the work to get there. Today we share yet another step towards this goal: we are opening our [Xata Discord Server](https://xata.io/discord) to everyone interested in joining in. “What does a discord server have do with DX?”, you may be wondering. I am glad you asked.

## The Vision

Having good DX means more than easy to use tools and readable code to us. It means being close to our users, it means knowing them, it means _being there_. Feedback like this from [selbekk](https://twitter.com/selbekk) gets us celebrating for days:

```tweet
1554155064880775170
```

From the Developer Relations team to the CEO, we are all there for you. Engaging, listening to your feedback and learning from how you use our product and what you want to build with us. We believe only our users can tell us what the _best serverless database DX in the world_ actually feels like, and we are here for it.

Creating a community also enables us to live up to another of our standards: **inclusiveness**. We want everyone to feel they belong in tech. And this server is our little corner where we will ensure everyone has the emotional safety to speak and be heard, to participate, to teach, and to learn.

## Sharing Knowledge

Within our server, sharing the knowledge you have (or just acquired) has a main role. Because it is easy to miss information in the endless scroll of channels where different topics intertwine and people get overwhelmed, we have created a set of channels with the main goal of encouraging people to participate and to allow what is said to linger for decent time. No FOMO (fear of missing out) around there.

We have different channels for:

- **showcasing** what you built
- ask for **help** for assistance in whatever you are building
- share tips via our twitter hashtag `#XataTips` (or a nice vibe with its sibling: `#XataVibes`)
- or maybe just **request access** because you had an awesome idea and we're in closed beta still

## Short Feedback Loop

On the first section, I mentioned we keep our ears up for users' feedback. Our discord server amplifies that in the way that we have announcement channels, where you will be able to hear first what we are working on and releasing. And, because we are all there, you can offer feedback in real-time as we share, or as we are implementing things.

- Do you want `.csv` export? Start a thread there.
- Is there a data type missing for your requirements? We want to know.

Drop a message, start a thread, let us know.

## Co-working space

As the cherry on top, we are a remote-first team. We know how the pandemic made everyone struggle and some are still in need of live interactions. So we have a special voice channel for **co-working**. Jump in the channel and someone may soon join in and chat for a bit. Maybe you're flirting with becoming a streamer, or you want some rubber ducking, or even you just want some coffee-break/watercooler kind of chat. Anything goes.

## Join us. Join us now!

In case you aren't familiar with discord, it's free. There isn't any information required, you open your account with an email and you can join our server with [this invitation](https://xata.io/discord). If something doesn't work well for you, reach out to us on [twitter](https://twitter.com/xata) or via `info[at]xata.io`. We'll do our best to help you sorted out. Say "hi" when you get there!


Come over and join the Xata Serverless Database Community on discord. It is a free and inclusive space for us to know each other better and learn together! We are also eager to learn from your experiences with our products.

Xata Serverless Database Community

Last week, Xata attended WorkerConf in Dornbirn, Austria where we presented a demo of our product. Here are our impressions.

## A Mountain Town with Astonishing Views

```tweet
1535534092577538049
```

The in-person conference was held in the Austrian alps, in a mountain town called [Dornbirn](https://en.wikipedia.org/wiki/Dornbirn). This town is possibly one of the most beautiful locations we've seen, with breakthaking vistas pretty much everywhere one can look. We arrived the day before the conference and had a hike with a few of the speakers and organizers. We were particularly moved by how eager they were to share their knowledge, experience, and culture around this town with us. It was very special indeed. 

## A Conference with Content that Delivers

The conference itself had an incredible range of [topics](https://www.alpine-conferences.com/worker-conf-2022/schedule/). WorkerConf is a backend-oriented conference, with more of an emphasis on server-side concepts, and it definitely delivered. The day started with a very thought-provoking and vision-casting talk by [Johannes Schickling](https://twitter.com/schickling), founder of [Prisma](https://twitter.com/prisma), about local-first application architcture which highlighted an event-sourced, local-first application he is building called [Overtone](https://github.com/overtone-app). 

```tweet
1535158943034527744
```

The talk highlighted problems with modern web applications, and proposed a solution that amplifies resilience, user agency, and a slew of other criteria vital to optimal developer and user experience.

The app was built with technologies we find particularly interesting. To name a few, 

- the [Riffle query engine](https://riffle.systems/essays/prelude/) that enables building data-centric apps with a reactive relational database,
- [Effect](https://github.com/effect-ts), a functional effect system for TypeScript with a rich standard library that makes code terser and safer at the same time, and
- [Tauri](https://tauri.studio/), a modern toolkit that helps developers make applications for the major desktop platforms. Kind of like [Electron](https://www.electronjs.org/), but implemented in [Rust](https://www.rust-lang.org/).

```tweet
1535191909718687744
```

The day continued with talks from industry leaders like [Lisi Linhart](https://twitter.com/lisi_linhart) from [Storyblok](https://www.storyblok.com/home), whose presentation beautifully married technical leadership with technical implementation, talking about how they built a globally distributed headless CMS and all of the interpersonal, cultural, and technical challenges that they solved along the way. 

Talk after talk, we were impressed and thankful for the content that was delivered, and the opportunities we had to connect with the community and discuss potential partnerships, collaborations, and integrations.

## A Team of Organizers who Care

```tweet
1535336636313178113
```

WorkerConf (and sister conference [AgentConf](https://www.alpine-conferences.com/agent-conf-2022/)) are organized by a small team of people who are passionate about community, interested to use technology and software to make the world a better place. This is essentially quoted verbatim out of the mouth of [Guntram](https://twitter.com/guntrambechtold), one of the organizers. As part of caring for the community, WorkerConf had an "activity day" after the conference where we hung out and had breakfast together. 

```tweet
1535606859616460801
```

After breakfast, those who were physically able _and interested_, had the opportunity to partake in alpine sports like mountain biking or trike riding. This was incredibly fun. We had a few falls and accidents, as one would expect, but there were no serious injuries and we all went home feeling closer and better than ever before.

## Conclusion

Ultimately, our feeling is that WorkerConf has accomplished all that it set out to do: celebrate Dornbirn, provide knowledge and a healthy exchange of ideas to developers, and form powerful connections. We are grateful for the opportunities afforded to us attending WorkerConf and we hope to see you in Dornbirn soon, either at AgentConf, or WorkerConf.

## PS

Was this blog post good? 

- If so, [tell us on Twitter](https://twitter.com/intent/tweet?url=https%3A%2F%2Fxata.io%2Fblog%2Fworkerconf-2022&text=This%20is%20a%20really%20great%20article%20about%20%23workerconf%202022%20from%20%40xata_io%20and%20worth%20a%20read!%20cc%20%40workerconf).
- If no, I'll go explore my career options as a bartender.

```tweet
1535173724055523328
```

Last week, Xata attended WorkerConf in Dornbirn, Austria where we presented a demo of our product. Here are our impressions.

WorkerConf 2022 Impressions

When showing [Xata.io](http://xata.io) (currently in private Beta) to people and saying that it is a “serverless database”, we occasionally get confused looks: how can a database be serverless? Doesn’t it need to store the data _somewhere_? This is a fair question because “serverless”, being the buzzword that it is, doesn’t have a definition that everyone agrees on. 

This is why we thought it’d be worth writing a blog post to document _our_ definition of a serverless database using straightforward language, in an attempt to bring a bit of clarity to the world.

First of all, hat tip to SQLite who was using the [term](https://www.sqlite.org/serverless.html) “serverless database” since before it was a buzzword, to mean something quite different: a database engine that runs in the process of the application. Rest assured, we’re talking about the buzzword serverless here. We’re Venture Capital (VC) funded after all. This version of “serverless” means a service–which _definitely_ uses servers–but those servers are abstracted away as much as possible by the said service.

## No Servers (to manage)

Yes, there are servers involved in our service. No, you don’t need to know about them. You don’t need to care how many of them there are, how much RAM they have, or if there are any low disk-space alerts raised recently, because those are totally and solely our problem (ugh). 

We’re taking the definition further than that to also include that you shouldn’t have to configure things like clusters, instances, nodes, backups, availability zones, monitoring, or what have you. Again, **those are our things to worry about**.

As a user, what you get is series of API endpoints that you can use to insert and update data, plus a set of APIs for querying, searching, and aggregating data. That’s it. Is there a fine print that comes with that? Oh yes, of course there is, and we’ll talk about that in the rest of the blog post.

Small side note: while not necessarily important for the definition of a serverless database, our API speaks HTTP and is therefore connectionless, it’s a lot easier to use from serverless functions because there’s no connection management to worry about.

## No Upgrades (that you have to worry about)

Databases, like most software, need occasional upgrading. The upgrading of databases in particular is usually annoying, dangerous, and can cause downtime. Some DBaaS (Databases as a Service) will take care of at least minor version upgrades for you, sometimes at the cost of having a maintenance window.

In our definition of serverless, there are no upgrades. Scratch that—just like with servers, there _are_ upgrades–it’s just that _we_ are the ones that do them. We do both minor and major upgrades in such a way that there’s no observable downtime to your users.

On the other hand, there might be versions of our APIs that you might want to upgrade to so you’re not _totally_ off the hook, but that tends to be a lot simpler by comparison.

## Durability and Availability (and SLAs)

When you send data to a cloud service like AWS’s S3 (Simple Storage Service), you don’t expect it to get lost with any sort of reasonable probability. That means that the service is highly durable. You also expect to be able to read the data at any time. That means that the service is highly available. Behind the scenes, in order to provide high-durability and high-availability, the service needs to store the data in multiple copies on different hard drives, and run servers in multiple availability zones. This also applies to databases.

If the database service is serverless, you shouldn’t have to worry about how many replicas and in which availability zones they are. Instead, as a user, you get an SLA (Service Layer Agreement) which details the durability and availability guarantees the service gives you, and what happens if those guarantees are not held. Usually, you get some money back.

We think high-durability and high-availability are important aspects of a serverless database service, so we include them in our definition and in all pricing plans, including the free tier. 

We’re still working on documenting our limits and SLAs, that’s part of why we’re still in closed beta.

## Performance and Scalability (and how they relate to pricing)

This is the part where we’d claim inter-planetary scale (to one-up our perceived competition) and “blazing-fast” performance for any query. And you’d just skip to the next headline because *of course we’d claim that*.

Instead, let’s have a discussion: what happens if a user sends a series of very inefficient queries quickly and in parallel?

There are database services out there that just don’t allow any queries that *can* be inefficient. That’s great from a service point of view, but their data model tends to be quite restrictive, usually restricted to some sort of key-value store.

Ours is not one of those. We have a pretty rich [data model](https://docs.xata.io/concepts/data-model) which allows complex queries and relations and everything, so there can be inefficient queries simply because they have to check a lot of data. Those inefficient queries are expensive for us to run, which is kind of a problem. In this case, there are two options:

- make the price per query proportionate to the work the query has done
- throttle queries and degrade the service predictably to the agreed capacity

We think both are valid strategies for a serverless database service, but in both cases you need to pay attention to the details. A drawback of the former is that it can become very expensive very quickly when you have a bad query. Some services allow you to set cost budgets, which are ideally checked in real-time. This replaces the risk of surprise bills with the risk of downtime when the budget limit is hit.

We are trying the second approach, which has its own challenges, but we think it’s ultimately the better model for the users. This model is less used today, but DynamoDB, one of the first serverless DBs, works like that. You can reserve a certain capacity, and you get throttled if you surpass it. In a way, it simulates what you would get from a “serverful” setup: if you haven’t allocated enough capacity, your performance will be degraded, but it won’t suddenly break on you. The difference is that we can make it a lot easier to add capacity or configure auto-scaling.

## Scaling to Zero (mostly)

A very cool aspect of the original serverless service, AWS Lambda, is that it scales down to zero in terms of pricing. That means that if a function is not used, it takes (next to) no resources and therefore costs nothing. This is great for personal projects but also for particular use cases even in the enterprise. This concept mostly translates to databases, but not quite–because even if the database receives no requests at all, it still needs to store the data.

In our definition of a serverless database, the compute costs should scale down to zero and costs should be incurred only if a lot of data is stored.

But there’s one more aspect to it: the infamous cold starts. Databases are typically not as quick to provision and boot as lambda functions. Some DBaaS will even suspend your instances after a few days of inactivity and you need to manually bring them up. 

In our definition, cold starts are very small and databases don’t get suspended after a period of inactivity that is measured in days. We’re hedging here a bit because we might have to eventually suspend databases as well, but we’d only do it after a very long period of inactivity: think years, not days.

## Summary (and no silver bullet)

Our definition of serverless databases can be summarised like this: **we take care of as many hard problems as possible so you don’t have to**. 

That includes scaling, upgrades, replication, schema migrations, backups, monitoring, and so on. But you don’t have to pity us for all those problems, we kind of like this type of work, and to be fair, we’re charging you for it–just less than what it would cost you to hire your own DB experts. 

That’s the value proposition of serverless databases as we see it, and we think it’s worth the hype: you’ll be able to run a business the scale of the next Airbnb, Uber, or Netflix, without DB experts and with a lot fewer “platform engineers” in general.

Is it a silver bullet, though? Do we take care of _all the problems_ that you can have with a database? Nope and nope.

For example, a bad data model or code that produces very inefficient queries will be either expensive for you or will feel sluggish for your users. Or maybe both. So it’s much better to fix the code. Also, our service, like any other, has various limitations, restrictions, and SLAs. What we promise is to document these clearly and explain them in plain language.

Being a serverless database and taking care of all of these problems for you is actually only half of Xata’s value proposition. The other half is about providing a top-notch developer experience and simpler developer workflows. But we’ll need to talk about that later, when we’re out of the closed beta. Until then, you can register for early access or follow us on [twitter](https://twitter.com/xata_io) for good vibes and so we can keep you in the loop.

What is a Serverless Database? There isn't a definitive definition that everyone agrees to, so this post documents our own definition, in an attempt to bring a bit of clarity to the world.

What We Talk About When We Talk About Serverless Databases

Last week, Xata had the privilege of sponsoring and MC-ing [JSConf Budapest](https://jsconfbp.com/). This is a blog post about our impressions from our involvement in this conference.

```tweet
1532810586433761280
```

## For the Love

JSConf Budapest is a community, not-for-profit conference that is run by a well-coordinated team of people who appear to deeply love the local JavaScript community and want it to flourish. Everything from the walkie-talkies team members use to communicate with each other, to the tears in their eyes after the conference concluded, demonstrate an intense sense of responsibility and care for the attendees, speakers, sponsors, and all involved.

```tweet
1532685490327015424
```

Because the conference is a _community_ conference with no explicit agenda for profit or marketing a specific product, the attendees were mostly everyday, _curious_ developers like you and me who were generally interested in JavaScript from web use-cases to embedded systems/Internet of Things use-cases to AI and everything in between, and this really showed at our booth.

![People vibing at the Xata booth](https://i.imgur.com/P9ARSHa.jpg)

We had people walk over with genuine interest in a "serverless database where people think data, not databases", and had the privilege of showing them the true power of Xata while answering common questions about how we're not Firebase and what our pricing is.


```tweet
1533112917842968577
```

Also, to no one's surprise, our inclusive all-gender swag–including women's cut shirts, earrings, and temporary tattoos–had people in awe, and others taking notes for their next booth. We're happy to have had this impact on the community and hope to see more earrings and inclusive swag in the future.

## All Are Welcome

There were people at the event for whom this was their first time in Europe: people from other countries who were awarded scholarships to attend the conference, including paid travel and accommodation. There were also child prodigies present, like [Connie](https://conniepocky.github.io/) who knows Racket, C, Clojure, JavaScript, Python and others–and is 13 years old. 🤯

```tweet
1532395564255551488
```

Because the conference chooses talks _completely anonymously_ based on the talk's title and description, there is no bias for _who_ is speaking: **it's purely about the ideas**. After a topic is chosen, _then_ the organizers learn who the speaker is, and make the necessary arrangements with them. With this model, people with great ideas from around the world–irrespective of their credentials or other backgrounds–get a platform.

This is how we got an amazing [talk](https://youtu.be/z4JnSGupxUs?t=20253) by [Lucky](https://jsconfbp.com/speakers/nhlanhla-lucky-nkosi) from South Africa about **5 Ways to Crash a Drone with JavaScript**. This was Lucky's first time out of Africa, his first time in Europe, and all of this was purely based on the content of his talk. We're deeply appreciative of this model of selecting speakers and would love to see more conferences follow suit.

JSConf Budapest makes it abundantly clear that JavaScript is for _everyone_, regardless of skin color, gender, age, and other variables. This welcoming nature even gave me my [first tech talk ever](https://www.youtube.com/watch?v=ONlLvZWXv1Y). This is something we especially appreciate about JSConf family events.

## JS the Community, not just JS the Language

```tweet
1533360616013971459
```

Some of the talks had absolutely no JavaScript code in them at all. And we found this to be a massive blessing because JavaScript is not just a programming language–it is also a community: a collective of people who–

- Solve problems with code
- Often communicate on teams 
- Go through pandemics together
- Are human beings, with human experiences
- Sometimes write CSS and HTML
- Other times fly drones

This conference is clearly about the community over the language, with special talks about mental health, emotional safety, documentation, and related topics where **it's not about the code, it's about us as JavaScript developers**. We found this particularly humanizing in recognizing we're more than just what we do with computers, and would love to see more of this in the wild.

## Togetherness

Organizing an in-person conference seems hard enough, but the organizers continued the party with more activities around the conference–like a community hike around the hilly region of Buda in Budapest.

```tweet
1533104847137939458
```

We visited a [child-operated railway](https://gyermekvasut.hu/en/home/), and continued walking and talking through the gorgeous Budapest hills with some nature therapy, food, and beautiful vistas.

All of us were somehow drawn to each other and stuck together as birds of a JavaScript feather. Wherever Xata people were, we were thankful that people felt safe, comfortable, and vibey, with some even explicitly mentioning “the Xata vibe”. We’re grateful to have this effect on the community and our peers, and actively continue to work on consistently bringing the best vibes. 

## "You're okay"

Multiple days of back-to-back social conference events _can_ take a toll on our health, with some of us needing downtime to recharge our social batteries either alone or in a smaller group. This organically happened on a natural and healthy cadence, and was acceptable by all.

```tweet
1532779856915742721
```

The organizers, attendees, and speakers all had a keen awareness and sensitivity to the needs of others, and accommodated everything a person needed to be happy and healthy. The condensed code of conduct, _"be excellent to each other"_, was very prevalent with people looking out for the best interests of each other.

## Conclusion

We see JSConf-family events as _the example to follow_ in the conference space, and JSConf Budapest, being the only JSConf in Europe, as the leader of the European portion of these. Year upon year (pandemic excluded), JSConf has continually provided immeasurable value, support, and passion to the JavaScript community, and we're eternally grateful to have been able to sponsor this conference and support the mission.

We plan to be back next year. 

Last week, Xata had the privilege of sponsoring and MC-ing JSConf Budapest. This is a blog post about our impressions from our involvement in this conference.

JSConf Budapest 2022 Impressions

The Xata engineering team values predictability at all levels of our codebase. This blog post is about how we handle API communications in a predictable way.

## Motivation

To us at Xata engineering, we consider spending time reverse engineering APIs just to know what kind of data is available to be neither fun nor valuable. Instead, through [OpenAPI](https://www.openapis.org/) we get some insights into the possibilities exposed by some APIs. However, why stop there? Reading an OpenAPI spec and manually writing code is… let's say, also not our thing.

Of course, [some](https://www.npmjs.com/package/openapi-to-ts) [libraries](https://github.com/swagger-api/swagger-js) exist to generate types from OpenAPI, but then we'd still need to manually connect them to [react-query](https://react-query.tanstack.com/) or similar fetchers, all while the generated types aren't very thorough in most cases. Even worse, in case of circular dependencies in a given OpenAPI spec, this becomes less of an option for us.

Given these constraints, we saw potential for innovation. _We had to do something!_ And this is why we started to craft [openapi-codegen](https://github.com/fabien0102/openapi-codegen).

The goals were clear:

1. Generate human-readable types, so we can review TypesScript files instead of a large OpenAPI YAML file.
1. Embed all possible documentation, so we can have access to the documentation directly through [IntelliSense](https://code.visualstudio.com/docs/editor/intellisense#:~:text=IntelliSense%20is%20a%20general%20term,%2C%20and%20%22code%20hinting.%22).
1. Generate ready-to-use React Hooks, with `url` and `types` baked into the component to avoid any human mistakes.
1. Be as flexible as possible: so far we can generate fetchers & react-query hooks, but let's not restrict the possibilities.

## Let's Play

To understand some of the superpowers this generator gives you, let's create a React application that consumes the Github API.

### Initialize the Project

For simplicity, let's use [`vitejs`](https://vitejs.dev/) to bootstrap our project!

```sh
$ npm create vite@latest
```

We'll select the framework `react` with the variant `react-ts`, this should give us a quick working environment for our application.

Now, let's start the magic!

```sh
$ npm i -D @openapi-codegen/{cli,typescript}
$ npx openapi-codegen init
```

- Select `Url`
- Paste the giphy OpenAPI url: https://api.apis.guru/v2/specs/github.com/1.1.4/openapi.yaml
- Enter `github` as namespace
- Select `react-query components`
- Enter `src/github` as folder

Execute `npx openapi-codegen gen github` and voilà!

### Inspect What We Have

Let's take a bit of time to inspect what we have in `./src/github`. The more interesting file is `GithubComponents.ts`, this is where all our react-query components and fetchers are generated and also our entry point. We have some generated types in `Github{Parameters,Schemas,Responses}`. These are a 1-1 mapping with what we have in the OpenAPI spec.

To finish, we have two special files: `GithubFetcher.ts` and `GithubContext.ts`. These files are generated only once and can be modified/extended to fit your needs.

In `GithubFetcher.ts` you can:

- inject the `baseUrl` (a default is provided)
- deal with authentication
- tweak how query strings are handled, especially if you have arrays

and in `GithubContext.ts` you can:

- tweak how react-query cache keys are handled (`queryKeyFn`)
- inject runtime values (hooks) to the fetcher (`fetcherOptions`)
- disable query fetching (`queryOptions`)

### Setup react-query

Of course, so far, we don't even have react-query installed, let's do this and setup our application.

```sh
$ npm i react-query
```

Add the `queryClient`

```tsx
// App.tsx
import { QueryClient, QueryClientProvider } from \"react-query\";

const queryClient = new QueryClient();

function App() {
  return (
    <QueryClientProvider client={queryClient}>
      <Users />
    </QueryClientProvider>
  );
}

function Users() {
  return <div>todo</div>;
}

export default App;
```

We can run `yarn dev` and see a white page with \"todo\".

### Start Fetching!

Let's try to fetch some users!

```tsx
import { QueryClient, QueryClientProvider } from \"react-query\";
import { useSearchUsers } from \"./github/githubComponents\";

const queryClient = new QueryClient();

function App() {
  return (
    <QueryClientProvider client={queryClient}>
      <Users />
    </QueryClientProvider>
  );
}

function Users() {
  const { data, error, isLoading } = useSearchUsers({
    queryParams: { q: \"fabien\" },
  });

  if (error) {
    return (
      <div>
        <p>{error.message}</p>
        <a href={error.documentation_url}>Documentation</a>
      </div>
    );
  }

  if (isLoading) {
    return <div>Loading…</div>;
  }

  return (
    <ul>
      {data?.items.map((item) => (
        <li key={item.id}>{item.login}</li>
      ))}
    </ul>
  );
}

export default App;
```

And voilà! Without knowing the API, just playing with the autocompletion, I'm able to fetch a list a users! 🥳 The types even give us a hint that `error.documentation_url` was a thing, quite cool, but let's see if this is actually working!

### Error Management

Let's refresh the page many times, until we reach the API rate limit 😅 Eventually, we don't see the error message, nor the documentation link.

This is where `GithubFetcher.ts` needs to be tweaked! Errors are indeed a bit trickier, as an error can be proper object back from the API or anything else (text response instead of JSON, or something less predictable if the network is down), so we need to make sure we received a known format in our application.

```diff
// GithubFetcher.ts
@@ -1,4 +1,5 @@
+import { BasicError } from \"./githubSchemas\";

@@ -43,8 +43,18 @@ export async function githubFetch<
     }
   );
   if (!response.ok) {
-    // TODO validate & parse the error to fit the generated error types
-    throw new Error(\"Network response was not ok\");
+    let payload: BasicError;
+    try {
+      payload = await response.json();
+    } catch {
+      throw new Error(\"Network response was not ok\");
+    }
+
+    if (typeof payload === \"object\") {
+      throw payload;
+    } else {
+      throw new Error(\"Network response was not ok\");
+    }
   }
   return await response.json();
 }
```

Here we need to make sure to validate the error format, since everything is optional in `BasicError` GitHub API and `BasicError` have a `message: string` property, I can safely throw an `Error`. And just like this, we have nice and type-safe error handling.

### Adding Some States

Of course, [React is all about state](https://twitter.com/TejasKumar_/status/1513123801994866689) isn't it? So… let's try!

```tsx
function Users() {
  const [query, setQuery] = useState(\"\");
  const { data, error, isLoading } = useSearchUsers(
    {
      queryParams: { q: query },
    },
    {
      enabled: Boolean(query),
    }
  );

  if (error) {
    return (
      <div>
        <p>{error.message}</p>
        <a href={error.documentation_url}>Documentation</a>
      </div>
    );
  }

  return (
    <div>
      <input value={query} onChange={(e) => setQuery(e.target.value)} />
      {isLoading ? (
        <div>Loading…</div>
      ) : (
        <ul>
          {data?.items.map((item) => (
            <li key={item.id}>{item.login}</li>
          ))}
        </ul>
      )}
    </div>
  );
}
```

Now I can search for users, and also reach the API rate limit way quicker! 😅 I guess this is a good time to introduce authentication handling in our little application.

### Authentication

To keep it simple for our demo purposes, let's take a GitHub developer token and store it in localStorage. **This is unsafe and you probably shouldn't do this in production**. If the token is not set, ask for it, if it's there, show the data and provide a disconnect button. We will need to access this `token` at runtime later, with this in mind, let's create a `Auth.tsx` file, that isolates our authentication logic.

```tsx
// Auth.tsx
import React, { createContext, useContext, useState } from \"react\";
import { useQueryClient } from \"react-query\";

const authContext = createContext<{ token: null | string }>({
  token: null,
});

export function AuthProvider({ children }: { children: React.ReactNode }) {
  const key = \"githubToken\";
  const [token, setToken] = useState(localStorage.getItem(key));
  const [draftToken, setDraftToken] = useState(\"\");
  const queryClient = useQueryClient();

  return (
    <authContext.Provider value={{ token }}>
      {token ? (
        <>
          {children}
          <button
            onClick={() => {
              setToken(null);
              localStorage.removeItem(key);
              queryClient.clear();
            }}
          >
            Disconnect
          </button>
        </>
      ) : (
        <div>
          <form
            onSubmit={(e) => {
              e.preventDefault();
              setToken(draftToken);
              localStorage.setItem(key, draftToken);
            }}
          >
            <p>Please enter a personal access token</p>
            <input
              value={draftToken}
              onChange={(e) => setDraftToken(e.target.value)}
            ></input>
          </form>
        </div>
      )}
    </authContext.Provider>
  );
}

export const useToken = () => {
  const { token } = useContext(authContext);

  return token;
};
```

Now, we can wrap our App with the AuthProvider, so the `token` is available in the entire application:

```tsx
// App.tsx
function App() {
  return (
    <QueryClientProvider client={queryClient}>
      <AuthProvider>
        <Users />
      </AuthProvider>
    </QueryClientProvider>
  );
}
```

and inject the `token` in `GithubContext.ts`

```diff
--- a/src/github/githubContext.ts
+++ b/src/github/githubContext.ts
@@ -1,4 +1,6 @@
 import type { QueryKey, UseQueryOptions } from \"react-query\";
+import { useToken } from \"../useAuth\";
 import { QueryOperation } from \"./githubComponents\";

 export type GithubContext = {
@@ -6,7 +8,9 @@ export type GithubContext = {
     /**
      * Headers to inject in the fetcher
      */
-    headers?: {};
+    headers?: {
+      authorization?: string;
+    };
     /**
      * Query params to inject in the fetcher
      */
@@ -36,14 +40,22 @@ export function useGithubContext<
   TData = TQueryFnData,
   TQueryKey extends QueryKey = QueryKey
 >(
-  _queryOptions?: Omit<
+  queryOptions?: Omit<
     UseQueryOptions<TQueryFnData, TError, TData, TQueryKey>,
     \"queryKey\" | \"queryFn\"
   >
 ): GithubContext {
+  const token = useToken();
+
   return {
-    fetcherOptions: {},
-    queryOptions: {},
+    fetcherOptions: {
+      headers: {
+        authorization: token ? `Bearer ${token}` : undefined,
+      },
+    },
+    queryOptions: {
+      enabled: token !== null && (queryOptions?.enabled ?? true),
+    },
     queryKeyFn: (operation) => {
       const queryKey: unknown[] = hasPathParams(operation)
         ? operation.path
```

And that's it! Now we have everything in place to create an amazing application around github API.

How is this working? Let's have a look of the generated `useSearchUsers` to understand.

```ts
export const useSearchUsers = (
  variables: SearchUsersVariables,
  options?: Omit<
    reactQuery.UseQueryOptions<
      SearchUsersResponse,
      | Responses.NotModified
      | Responses.ValidationFailed
      | Responses.ServiceUnavailable,
      SearchUsersResponse
    >,
    \"queryKey\" | \"queryFn\"
  >
) => {
  // 1. we retrieve our custom auth logic
  const { fetcherOptions, queryOptions, queryKeyFn } =
    useGithubContext(options);
  return reactQuery.useQuery<
    SearchUsersResponse,
    | Responses.NotModified
    | Responses.ValidationFailed
    | Responses.ServiceUnavailable,
    SearchUsersResponse
  >(
    queryKeyFn({
      path: \"/search/users\",
      operationId: \"searchUsers\",
      variables,
    }),
    // 2. the custom `headers.authorization`, part of `fetcherOptions` is passed to the fetcher
    () => fetchSearchUsers({ ...fetcherOptions, ...variables }),
    {
      ...options,
      ...queryOptions,
    }
  );
};
```

## What About Query Caching?

One of the key features of react-query, is to have a query cache. By default, openapi-codegen provides you with a key cache that fits the URL scheme.

For example:

`useSearchUsers( { queryParams: { q: \"fabien\" } }` will produce the following cache key: `[\"search\", \"users\", { q: \"fabien\" }]`.

So if I want to invalidate the users list, I can call:

```tsx
import { useQueryClient } from \"react-query\";

const MyComp = () => {
  const queryClient = useQueryClient();

  const onAddUser = () => {
    queryClient.invalidateQueries([\"search\", \"users\"]);
  };

  /* ... */
};
```

Note: This is just a default behavior, if you have more specific needs, you can always tweak `GithubContext#queryKeyFn`.

## Help & Feedback

This project is still relatively early stage (but it strictly follows semver, so if we break the API, you will know it!). We're always happy to have feedback and help you if you have any questions. If you'd like to browse the code or take it for a spin, be sure to check out the [repo on GitHub](https://github.com/fabien0102/openapi-codegen).

Happy coding folks!

The Xata engineering team values predictability at all levels of our codebase. This blog post is about how we handle API communications in a predictable way.

From OpenAPI to Type-Safe React-Query Hooks

Xata is a fully remote team with employees in Spain, Germany, Andorra, Belgium, and the US, with plans to add more countries soon. To compensate for our mostly Zoom-based work lives, we’re meeting in person a few times per year for an offsite in a nice location.

Last week, we had our first such offsite in Marbella, Spain, and here is how we did it and what we have learned from it. 

Fair warning: the photos in this post might make you hungry.

## Preparation

First, a couple of months in advance, we chose the dates and the location. For the dates, we wanted it as soon as possible, but also when we expected the COVID-19 cases to go down, reducing the chances of travel restrictions. A couple of people on the team will go on parental leave soon and others had planned vacations, so we took those into account as well. 

We decided to do the event Monday to Friday, including traveling, so that people can spend time with their friends or families on the weekends and don’t have to change plans.

![Marbella from above](/images/blog/marbella/vacaciones-en-marbella.jpg)

For the location, we chose a place with nice weather that is well-connected to make traveling to/from it simple. We have a lot of colleagues in Spain, so the south of Spain was a good choice because many could arrive by train. The nearby Málaga airport made it convenient for the others.

We found a local [travel agency](https://www.facebook.com/ViajesCarrefourSM/) to help us with booking the hotel, arranging the transfers, fun activities, and so on. This proved to be an extremely good choice and resulted in a surprisingly well-run event for a company of our size.

## Agenda

The game plan was:

- **Monday:** Everyone arrives.
- **Tuesday:** We review and discuss the way we work, our strategy, and our messaging.
- **Wednesday:** We review and discuss our roadmap and plans.
- **Thursday:** We visit the region and have fun together.
- **Friday:** We travel back.

Like in a conference where the most valuable track is the “hallway track”, more important than the agenda was what we discussed during breaks, during dinners, or simply on the bus. So we made sure the schedule had plenty of breathing air for us to know each other.

## Monday: arrivals

For many of us, this was the day that we met in person for the first time, despite working together for several months already. This made everyone very excited to meet each other. When we reached the hotel, the Xatafly — which is how we nicknamed our logo — was there to welcome us.

In the evening we had our first dinner with the whole team, and we learned quickly that the food in Spain is incredible and that we’re getting plenty of it.

![Dinner on the first evening](/images/blog/marbella/IMG_6425.jpg)
![Meeting in the Berlin airport](/images/blog/marbella/IMG_6399.jpg)
![Xatafly at the hotel](/images/blog/marbella/IMG_1148.jpg)
![Calamari](/images/blog/marbella/PXL_20220328_180302157.jpg)


## Tuesday: Team, strategy, and messaging

We started by discussing our [Series A](https://xata.io/blog/xata-series-a-announcement) announcement and what it means for our overall strategy. We then did a team retrospective which was a very open discussion and gave us a lot of food for thought. Speaking of food, the frequent “coffee breaks” included a lot more than coffee, more like a full buffet. It’s ok, we can diet the week after.

![Monica and Yannis](/images/blog/marbella/DSC04311.jpg)
![Tuesday in the meeting room](/images/blog/marbella/DSC04288.jpg)
![Coffee break](/images/blog/marbella/PXL_20220329_091206311.jpg)
![Xata team members](/images/blog/marbella/DSC04325.jpg)

After lunch, we discussed our messaging, which included a very fun and productive brainstorming on our tagline. Towards the end of the day, we took on a couple of technical topics, because in the end, we’re engineers and that’s how we relax.

Dinner was again fabulous.

![Dessert](/images/blog/marbella/PXL_20220329_202748882.jpg)
![Dinner on Tuesday](/images/blog/marbella/IMG_6451.jpg)

## Wednesday: Planning and roadmap

A few brave people started the day early with a workout at the hotel gym. The rest of us found reasonable excuses to not do that.

![Xata workout](/images/blog/marbella/IMG_1190.jpg)

In the morning we did a high-level planning session. We went through our roadmap towards our first GA release, made sure it was complete, and re-ordered items according to their relative priority. After lunch, we have discussed our pricing model, which was about as fun as it sounds.

With the weather becoming really nice, we decided to stay outside for the end of the meetings day. This made the discussions more casual and less organised, but not less valuable.

![Xata founding team](/images/blog/marbella/IMG_6474.jpg)
![Xata chilling in the pool](/images/blog/marbella/PXL_20220330_144107304.PORTRAIT.jpg)
![Tudor and Alex](/images/blog/marbella/DSC04339.jpg)
![Xata team in the sun](/images/blog/marbella/DSC04352.jpg)
![Xata team making butterfly signs](/images/blog/marbella/IMG_6473.png)

In the evening, for a change, we cooked our own food! Our amazing teacher Jose showed us how to make famous Spanish dishes: Gazpacho, Bacalao al Pil-Pil, and, of course, Paella. One Paella got too much rice, one too much water, and one was just about right. It didn’t really matter, we all had a great time. Thanks, Sangria.

![Yannis likes it](/images/blog/marbella/DSC04492.jpg)
![Tejas making Paella](/images/blog/marbella/DSC04446.jpg)
![Team one cooking](/images/blog/marbella/DSC04431.jpg)
![Bacalao al Pil-Pil](/images/blog/marbella/DSC04425.jpg)
![Team two cooking](/images/blog/marbella/DSC04470.jpg)
![Team three cooking](/images/blog/marbella/DSC04477.jpg)

## Thursday: Fun day

We started the day by walking down to the port of Marbella for a Catamaran trip. The water was choppy which means that we didn’t see dolphins as we were hoping but the waves kept us entertained.

![Xata team lounging on the boat](/images/blog/marbella/IMG_1242.jpg)
![Alex chilling](/images/blog/marbella/DSC04501.jpg)
![Yannis and Tejas on the boat](/images/blog/marbella/IMG_6593.jpg)
![Xata team on the boat](/images/blog/marbella/IMG_6653.png)
![Carlos on the boat](/images/blog/marbella/IMG_1244.jpg)

After another delicious lunch, we visited Cueva del Tesoro and the beautiful city of Mijas for a relaxed afternoon and evening.

![Burro Taxi Mijas](/images/blog/marbella/DSC04624.jpg)
![Xatacat](/images/blog/marbella/DSC04643.jpg)
![Part of the Xata team in Mijas](/images/blog/marbella/IMG_1290.jpg)

## Friday: travel back home

After dinner on Thursday evening, we hugged and said goodbye, because many of us had flights or trains early in the morning on Friday. Others could stay a bit more and enjoy the beautiful weather.

![On the beach](/images/blog/marbella/Image_from_iOS_(3).jpg)
![Malaga city center](/images/blog/marbella/PXL_20220401_113036856.jpg)

## Learnings and conclusions

- There was a lot of laughter during the off-site. Coming after the long pandemic and the associated social distancing, it reminded us just how important laughing together is.
- As a remote-first company, we do a lot of work async and in writing. Meeting each other for a few days did wonders in associating a friendly voice and tone with future written comments.
- A hotel meeting room is not the best setting for casual conversations, so don’t spend too much time in it. Casual chats during the breaks brought us the most interesting ideas.
- Finding a good local travel agency makes organizing a lot easier.
- We’re getting back more energized, motivated, and with a clearer plan.
- Did we mention the food in Spain is amazing?

![Xata wants you](/images/blog/marbella/Studio_Project.jpeg)

If you’ve got FOMO from the pictures above, why don’t you [join us](https://join.com/companies/xata) on the next offsite? We’re working on a [Serverless Database](https://xata.io/) with great UX and DX, meaning that we have lots of interesting challenges across the board: backend, frontend, platform, DevRel, UX, and more. We’re looking forward to chat with you!

Last week we did our first Xata off-site, in Marbella, Spain. This is a photo-blog post about how it went and what we've learned from it. Warning: this post might make you hungry.

Offsite for Our Fully Remote Team

[Git](https://git-scm.com/) has become a tool that developers can't live without today. It has evolved beyond its initial role as just a version control system to support workflows throughout the entire development process. Part of this progression is purely a result of Git's inherent robustness. Another part of this success, though, is credited to the vast ecosystem that has been built around Git.

One of the Git features that enables and inspires this type of large-scale integration is its amazing branching mechanism. It has allowed a large number of contributors to collaborate on a single project while using automated workflows to guarantee error-free delivery.

At the same time, Git branching has given software development a layer of predictability. What else would you call being able to test and preview how your code changes affect the system _before_ making them permanent after all? That's why even though it's a simple concept, branching creates such a huge difference in the way we approach a development process.

## How does branching work?

Git branches are not that different from their namesake, tree branches. They stem from the main body of the codebase to follow a development path independent from the direction of the rest of the codebase. They give you an environment to work on new features, fixes, or any other task, isolated from the changes made by other developers or ongoing implementations. Often, branching is used to maintain the production/staging code in the main branch while the development is carried out in feature branches.

![Feature Branches in Git](https://c-v.sh/elmysurfperch.png)

Unlike tree branches, though, Git branches are capable of merging with another branch after a while in isolated development. It allows you to add fully-implemented features in another branch to the production branch, after separately running tests to evaluate their functionality and performance.

![Merging Feature Branches into a Main Branch in Git](https://c-v.sh/unforgeddiddlers.png)

### Branching & Pull Requests

Another Git feature that goes hand-in-hand with branching to enhance the predictability of the development process is _pull requests_. It's especially powerful today because of how some tools in the Git ecosystem—like [GitHub](https://github.com/), [Bitbucket](https://bitbucket.org/product), and [GitLab](https://about.gitlab.com/)—have built additional functionality around pull requests to promote better collaboration among developers and continuous integration.

With pull requests, or PRs as they are often called, you can send a request to merge a branch in your own repository with a branch in another repository. Usually, this mechanism is used to merge the changes made in a feature branch of a forked repository with the parent's production branch.

You can use PRs to not only review the code that is being pushed to the main branch but also run automated workflows to test or preview the incoming changes. It helps you ensure that these changes won't cause any unexpected behavior in the application or trigger errors. By combining the right branching and PR strategies, you can collaboratively develop applications that operate just as you want, leaving little room for unpredictable outcomes, especially while they are in production.

[Vercel](https://vercel.com/)'s [deploy preview](https://vercel.com/features/previews) feature is one of the latest tools that takes advantage of the branching and pull request mechanisms to extend the predictability Git offers beyond the code level.

### Deployment Previews with Vercel

Vercel allows deploying websites directly from a Git repository hosted on GitHub, GitLab, or Bitbucket. Once you declare the repository's production branch, every time you commit changes to it through merging pull requests, Vercel automatically rebuilds and deploys the website to its platform.

Besides these regular deployment capabilities, however, Vercel contains a special feature that allows you to preview how the site will function with the incoming changes _before_ committing them. Vercel integrations with Git-based platforms automatically deploy PR or merge requests onto a temporary preview domain, where you can test whether the modifications are behaving as expected without exceptions. These are called **preview deployments**.

Once you are satisfied with the modified functionality, you can trigger a production deployment in the production branch with the new commits.

## Branching meets databases–is it a reality?

Git branching gives developers a lot of freedom to make changes to a codebase during an ongoing development process. Vercel has extended this freedom to the deployment level with its Git integrations. But does the luxury of branching extend to the database level? The answer to this question would have been a resounding no a couple of years ago. But with the product we are now developing at Xata, a database that supports Git-like branching and merging mechanisms is no longer just a wishful thought.

Xata is a serverless database delivered as a simple REST API ideal for developing web apps. More importantly, it allows you to create separate branches to isolate new feature implementations from the rest of the database and merge them back whenever you're ready to deploy the changes. Naturally, data is not copied between branches for security.

With Xata's support for creating database branches, you no longer have to maintain several databases at a time for testing and development purposes. All the development and testing processes can be carried out via a single database, without changes in one branch affecting the state of production or other existing environments. It gives you the chance to employ an end-to-end branch management system in your development process.

[Xata branches](https://docs.xata.io/concepts/branches) enable you to modify and create schemas, test how well they perform, use Vercel deployment-like previews before merging the changes, and finally, migrate the updated schemas to deploy them to your application. It executes all these tasks as rolling migrations with zero downtime.

With Xata added to your Git-powered development stack, you can foresee how your development decisions affect the entire product before making irreversible changes. It gives more predictability to your development process, from beginning to the end, without leaving much room for costly mistakes.

Let's explore the benefits of branching in developer workflows, and how Xata brings these benefits to data infrastructure.

How Branching Brings Predictability to Web Development

Today, we’re excited to announce that **we’ve successfully raised a Series A investment of $30M, co-lead by Redpoint and Index Ventures**. This will help us build the go-to serverless database that engineers have been waiting for, with powerful features including a fully serverless design that provides automatic scaling, ACID transactions, database branches, zero-downtime migrations, and a full-text search engine with analytics. All of these features are available in conjunction with a beautiful and natural user interface (UI) that feels like a simple spreadsheet, but is capable of inter-table relationships and more.

With this investment, we also formed a new board of directors. We are incredibly proud to announce that Xata’s board is fully composed of women. Myself (Monica Sarbu) as CEO and founder, [**Erica Brescia**](https://www.linkedin.com/in/ebrescia) from Redpoint as the director of the board, and [**Erin Price-Wright**](https://www.indexventures.com/team/erin-price-wright/) from Index Ventures as a board observer. Erica was recently COO at [GitHub](https://github.com) and co-founder of [Bitnami](https://bitnami.com/), while Erin has worked previously at [Palantir](https://www.palantir.com/) and serves on the boards of companies like [Strapi](https://strapi.io/) and [ClickHouse](https://clickhouse.com/). Erica and Erin are some of the most capable and impressive people we have met, and we are beyond excited to be working closely together towards creating a game changing database experience for the mass market.

## What People Are Saying

We were able to successfully raise this funding round because a lot of people believe in Xata. Here's what some people are saying.

> Monica is a truly special, powerhouse of a leader and her vision for delivering a serverless database that combines the simplicity of a spreadsheet with robust, globally distributed and infinitely scalable infrastructure is incredibly exciting. We have seen from our investments in Snowflake, Cockroach Labs and others just how big businesses like Xata can become with the right product and leadership. We’re thrilled to be partnering with Monica and team to build Xata. --Erica Brescia (Managing Director, Redpoint; ex-COO GitHub and Founder of Bitnami)

> We are delighted to be working closely with the team at Xata as they shape their vision. Monica Sarbu models AWS’s builder DNA and we support her commitment to driving diversity in tech. --Tanuja Randery (Vice President, Amazon Web Services EMEA)

> When I started Vercel and Next.js, I dreamt of a data layer that’d be friendly for devs and non-devs alike. It’s finally here: Xata delivers on a great developer experience and a gorgeous admin UX. --Guillermo Rauch (Founder & CEO, Vercel)


## Xata's Origins

The idea for Xata formed as we were working on a [non-profit](https://tupu.io) centered around mentoring underrepresented communities in tech during the COVID-19 pandemic. As we wanted to minimize our operational overhead, we chose to use Jamstack and serverless technologies. **This worked well, especially thanks to platforms like Netlify and Vercel**, but we also needed a database–and that part was not as easy.

We eventually chose Airtable for the non-profit which gave us a simple data API and took care of most concerns. It also gave us a very nice UI that we could use as an internal tool to do the matching between mentors and mentees. **This worked well for a while, but we’ve found Airtable to be rather limited** in the amount of data we could store, as well as lacking more developer-centric functionality like transactions and constraints, which would greatly benefit our non-profit.

After speaking with a number of developers, we were surprised to find out that using Airtable as the main database is more common than we thought and the systems that people build on top of Airtable can get quite complex. **The room for improvement in the database space from a usability and developer experience standpoint is what inspired us to start Xata.** Our purpose as a company is to make databases a joy to use for everyone building an app today, not just a select few.

We’re currently in private beta and are inviting developers from an internal waitlist to try Xata out. If you have a particular use case for which you’d like to try Xata, please [reach out](https://twitter.com/xatabase) to us.

![Xata stands with Ukraine](/images/blog/series-a/xata-ukraine.png) 


While we are extremely excited about our growth as a company, we cannot finish this blog post without acknowledging the current heartbreaking humanitarian crisis in the Ukraine. We are keeping an eye on the situation and will seize every opportunity we encounter to support truth and justice in this time. We are making personal donations to organizations that are helping those most affected by the war.

Today, we’re excited to announce that we’ve successfully raised a Series A investment of $30M, co-lead by Redpoint and Index Ventures. More in the content of this post.

Xata Raises $30M to Launch a Serverless Database With the Usability of a Spreadsheet

Compared to the static web pages of the early days, today’s web has come a long way in terms of content served to users and the technology used. Websites of the past with a handful of pages delivered via a [Content Delivery Network (CDN)](https://en.wikipedia.org/wiki/Content_delivery_network) have expanded to using backend servers, databases, dynamic content, and myriad other technologies to provide users a better experience while solving problems that arrive with ever-increasing demand. With all these changes, **modern web development has grown to be an incredibly sophisticated field**.

Perhaps, it’s a reason why the emergence of Jamstack, which talks about returning to the simplicity of the early days, has become a baffling concept to some developers. Is it possible to bring back the days of delivering static web pages via a CDN when the web has already gotten so complex? How can it provide the performance and functionality of already established stacks like LAMP and MERN (more about these below).

To find the answers to these questions, we have to look a little deeper into what Jamstack is and how it compares with traditional stacks like LAMP and MERN on different fronts.

## What is the Jamstack?

The _JAM_ in Jamstack refers to combining **Javascript, APIs, and Markup** as core technologies to develop web apps. However, the term has a more specific meaning which can be summarized like this:

> Jamstack is the technology stack used to deliver fast and secure web apps to users by pre-rendering pages and serving them from a CDN, eliminating the need to manage servers.

As mentioned at the beginning of the article, Jamstack is an attempt to bring back the simplicity of the static web by using two main principles.

The first is **prerendering**. It refers to the idea of generating as much web page markup as possible during the site build time so that the content can be directly served via a CDN. It eliminates the need for setting up backend servers and other complex configurations associated with it—like managing clusters/nodes, implementing security measures, and caching—to serve dynamic content at user request. As a result, Jamstack websites can load faster and serve a better experience to users. The removal of backend from the picture also makes Jamstack websites more secure simply because it exposes fewer components to the outside world.

The second main principle behind Jamstack is **decoupling**. With the introduction of powerful frontend frameworks to the market, the line between the frontend and backend is gradually becoming blurry. Inevitably, this has led to increased complexity in frontend development. In the Jamstack, decoupling looks to reestablish this clear divide between the frontend and backend by outsourcing backend tasks to third-party APIs and services.

Preferring pre-rendered web pages over dynamically-rendered ones, however, doesn’t make Jamstack a bad candidate for building dynamic websites. This is where the Javascript and API parts of the JAM backronym become important. Jamstack websites use third-party APIs to dynamically load data through requests made via client-side Javascript or serverless functions.

### Jamstack Ecosystem

The reason why Jamstack has been able to bring back the simplicity of earlier websites, without losing the advanced functionality supported by traditional stacks, is the ecosystem that surrounds it. The components of this ecosystem have grown independent from Jamstack in the last decade. But now, combined together, they have formed a stack that lets us build faster, secure, cheaper websites.

Jamstack ecosystem primarily consists of the following components:

- CDNs: Recently, many CDN providers catering to Jamstack websites have emerged, like [Netlify](https://netlify.com) and [Vercel](https://vercel.com). Deploying a website to these CDNs is simple as connecting the hosted Git repository. The integrated CI/CD workflows of these platforms automatically trigger builds whenever new commits are made.
- Static Site Generators (SSG): The popularity and abundance of SSGs like [Next.js](https://nextjs.org/), [Hugo](https://gohugo.io/), [Gatsby](https://www.gatsbyjs.com/), and [Nuxt](https://nuxtjs.org/) in the past couple of years support pre-rendering web pages at build time.
- Third-party APIs: Mass availability of third-party APIs like [Auth0](https://auth0.com/) for authentication, [Stripe](https://stripe.com/en-de) for processing payments, and [Algolia](https://www.algolia.com/) for searching allows us to outsource previously backend-bound tasks and abstract their complexity.
- FaaS (serverless functions): Serverless functions help Jamstack websites to integrate heavy logic without calling for backend servers.

### How the Jamstack Makes Web Development Better

- Built with mostly pre-rendered markup, Jamstack sites load faster on the client-side.
- Having no backend servers to protect makes the sites more secure.
- Because they are stateless, the websites can scale on-demand without needing special server configurations or code changes.
- You often only have to pay for the resources you use and nothing more.
- With no backend to configure, you can host your entire website on Git and impose end-to-end version control.

To understand this point even better, let’s compare Jamstack against two of the most popular traditional stacks on the web, LAMP and MERN.

## Jamstack vs. LAMP stack

Proven against the test of time, LAMP has been a go-to technology stack of web developers for more than a decade. Even the most famous app on the web, [WordPress](https://wordpress.com/), runs on LAMP. The four letters in LAMP represent four open-source components that make essential contributions to running dynamic web apps.

- **L**inux operating system.
- **A**pache web server for processing client requests and HTTP routing.
- **M**ySQL database for data storage and querying.
- **P**HP for handling internal logic and serving dynamic content.

Today, LAMP is supported by a mature and extensive ecosystem that supports building robust dynamic websites for any kind of use case. Its completely open-source architecture gives developers full control of their sites with no fear of [vendor lock-in](https://en.wikipedia.org/wiki/Vendor_lock-in).

Despite these positives that have popularized LAMP in the development community, when compared against newer technology stacks like Jamstack, LAMP comes with too much complexity and too little flexibility for modern-day websites.

- LAMP stack has a steep learning curve when learning to configure the backend for everything to work together. Beginners, especially, find this process too complex and cumbersome.
- Dynamically processing and rendering content at request time reduces the app performance from the user's perspective.
- LAMP doesn’t scale easily with high traffic without manual intervention to add more server power, caching layers, and optimized code.
- With all the backend configurations that cannot be automated, implementing CI/CD workflows for the entire website is not practical.

## Jamstack vs. MERN stack

Compared to LAMP and Jam, MERN is an open-source web development stack still in its early days. However, due to its relative simplicity and scalability, MERN has become immensely popular among developers in the past couple of years. It provides a full-stack solution to web development that consists of four main components:

- **M**ongoDB, the NoSQL database
- **E**xpress.js, the backend development framework
- **R**eact, the frontend development framework
- **N**ode.js, the Javascript runtime environment for backend

The introduction of Node to the backend gives developers the comfort of working with Javascript on both frontend and backend without having to learn another language. The use of Express and React frameworks also abstract away some of the complexity of backend and frontend configurations compared to the LAMP stack.

However, MERN is still tied to the burden of backend, namely dynamically processing content at request time at the cost of site performance. It’s also an ill-fit for working with highly relational data. Though replacing the non-relational MongoDB in this stack with a relational database is a solution to this problem, Node still lacks ORMs that are advanced enough to support complex queries. This results in developers having to manually write SQL queries, increasing the complexity of the development process.

## Jamstack and Databases

The original philosophy of Jamstack recommends outsourcing database-bound tasks in a web app. Yet, this is not something that’s always attainable or practical. It’s easy to find yourself in situations where you need control of how your data is stored and accessed for functional, legal, or security reasons. But using a traditional database with Jamstack diminishes its initial benefits like increased performance, security, zero-configuration set up, and end-to-end version control.

That’s why we developed Xata to bring the best of both worlds to developers. As a connectionless relational database, Xata is delivered as a lightning-fast and highly available API that Jamstack websites can query from the client-side or through serverless functions. **It’s the first-ever database to support the development workflow with version control features like branching, previewing, testing, and merging changes.** With these features, Xata gives back the ability to fully automate the CI/CD workflow as Jamstack promises.

Xata also removes a lot of inconveniences tied to traditional databases like scaling, caching, maintenance, security updates, and downtime handling, making it accessible to even those without development experience to build and deploy their own apps. **With the addition of Xata, Jamstack becomes an even more powerful way to build websites that makes the lives of users and developers a lot easier.** It takes you back to the simplicity of the early days of web, while retaining all the breakthroughs this field has made in the past two decades to serve users a better experience.

We're currently in private beta, so if you'd like to get started with Xata today, sign up on the [home page](/), or [@-us on Twitter](https://twitter.com/xatabase) and we'll be sure to hook you up. People with immediate use cases and nonprofits tend to get early access relatively easily.


This post is an in-depth look at the Jamstack, how it compares to other alternatives, and how Xata aims to complete it by providing a serverless database.

Jamstack: a Deep Dive

Today we're announcing a new open source package that takes feedback to the next level: introducing [`@xata.io/screenshot`](https://github.com/xataio/screenshot).

![Demo](/images/blog/screenshot/docs.gif)

At Xata, we're committed to serving our users **the smoothest developer experience** any managed database service has ever offered. We're well aware that what's frictionless to _us_ may not be frictionless to _everyone_, and so we're taking a highly feedback-based approach to doing this.

## Inspiration

We recently rolled out our product to our first few early-stage adopters, including our founders' nonprofit organization [Tupu](https://tupu.io/) and [Saddleback Church Berlin](https://saddleback.berlin). As we continue to roll out Xata, we recognize the importance of providing users an instant way to share realtime feedback with us while capturing the exact context they're in–and so, we wrote a library.

## Enter `@xata.io/screenshot`

Our [screenshot library](https://github.com/xataio/screenshot) wraps the native [MediaDevices](https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices) web platform API with a more intuitive and straightforward abstraction that allows users to take a screenshot from their computer, which they can then include in their feedback to us.

It works like so:

- You call a function, `await takeScreenshot()`
- the user is asked for permission to view their screen
- they are asked to select _what_ to screenshot: the current tab, another window, another tab, or their entire screen
- once they confirm, the screenshot is captured and you get a Base64-encoded image of their screen

You can then save this string as an image, or forward it to a hosting service like [S3](https://aws.amazon.com/s3/), [c-v.sh](https://c-v.sh/) or similar after processing.

### Diving Deeper: The Technical Details

The web platform provides a powerful set of APIs under the `MediaDevices` class. One of these APIs is `MediaDevices.getDisplayMedia`. This API is what prompts a user to select and grant permission to capture the contents of a display or portion thereof. One might think of this as `window.alert` on steroids.

This function returns a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) that resolves into a [`MediaStream`](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream): an interface representing a stream of media content. In this case, the stream contains the portion of the display they've chosen to share.

Our library does the following:

- gets this video stream
- creates but does not visibly render a `<video>` element in memory
- sets its `src` to this stream
- waits for the video feed
- waits for a frame of the current screen to be painted on the video
- paints this frame onto a `<canvas>`
- serializes the canvas to a base64 string using `HTMLCanvasElement.toDataURL()`
- returns the string

The [source code](https://github.com/xataio/screenshot/blob/main/src/takeScreenshot.ts) for this library is fully open and available on GitHub and we're proud to add it to our soon abundant repertoire of open source libraries and helpers.

## How It's Used in Xata

With the exception of this website (a new one is on the way soon!), all Xata public-facing properties expose a little feedback button where users can submit feedback at any time during their usage of the property.

This feedback button opens a modal where they can type feedback, and also includes a little checkbox to include a screenshot. When they click this checkbox, this library is invoked. A full demo of this can be experienced on our [documentation site](https://docs.xata.io).

## Conclusion

This is just the beginning. Feel free to check out the [source code of this library on GitHub](https://github.com/xataio/screenshot). At Xata, we're committed to solving entire classes of problems developers face and providing frictionless solutions to them–usually in the database space, but sometimes in adjacent areas as we see opportunities. In the coming weeks, we're excited to roll out more exciting open source software.

To stay up to date, please be sure to [follow us on Twitter](https://twitter.com/xatabase), and [sign up for our mailing list](/) on the home page. Also–we're hiring! If you'd like to join us and consider yourself competent in Golang, TypeScript, or Developer Relations, [apply here](https://angel.co/company/xata-inc) or @-us on Twitter.


We have created a TypeScript library for the web that allows users to instantly capture and share screenshots. More technical details inside.

New Open Source Library: Announcing @Xata.Io/Screenshot

If you haven't heard of us before, it's because we haven't launched yet. We're busy building the product, but we don't believe in stealth mode, so we'd rather talk to you about what we're building and why.

Databases today are still too complicated and they leave too much for the developer to worry about. Even with the managed databases services offered by the big Cloud providers, as a developer, you still need to worry about things like:

- maintenance windows and downtime for security upgrades
- configuration of nodes/instances, replication, and scaling
- replicating the data into a different analytics or search engine
- caching with an external system, and dealing with cache invalidation
- applying schema migrations and avoiding downtime during migrations
- setting up a monitoring system for basic things like disk usage
- implementing access controls in the application code
- working around the lack of transactions, or joins in the application code
- polling for changes
- dealing with the lack of schemas or the lack of rich types, like arrays, nested objects, or attachments
- dealing with clumsy admin interfaces and no easy way to add test data

The list is so long, and each item on the list is its own deep rabbit hole, that it is even hard to grasp the complexity.

Granted, not all of these problems apply at the same time. Some existing services are good at solving a subset of the above. For example, DynamoDB is fully serverless so you don't have to worry about upgrades, nodes, scaling, or security patches. But it has a very rigid data model that doesn't support flexible queries so you often have to pay for the convenience by making your application code more complex.

A service like AWS Aurora for PostgreSQL/MySQL, on the other hand, brings you the flexibility of the SQL language and the rich open-source ecosystem of ORMs. While it is a huge leap over running MySQL or Postgres yourself, Aurora is not truly serverless, which means that you still have to worry about how many replicas you need, upgrading between major versions, downtime while patches are applied, and so on. Then, if you need to sync your data in an analytics or search engine, you need to learn about logical replication, you're probably going to need a queuing system, which requires its own monitoring system. Not to mention having to choose a migration tool, an admin UI, a connection pooler, etc.

This is way too complex for beginners. And while more experienced developers have dealt with these things for a long time, many would rather outsource these concerns. For these reasons, developers sometimes pick No-Code/Low-Code tools like Airtable to store their data. Airtable's intuitive and rich data types and the excellent UI make it easy to add data and visualize it in different ways. It has search and analytics and views built-in. A simple RESTful API makes it easy to query and filter your data. Oh, and it's free if you don't have a lot of data.

But Airtable wasn't built to be consumed by applications. For example, its rate-limiting hits very quickly, at just five requests per second as of today. It also has limitations in terms of the number of rows in a table and lacks consistency tools that simplify the code, like transactions and constraints.

**Is it possible to make a database service that is as simple to use as Airtable, yet scalable, robust, and safe as the traditional databases?** Yes, there is really no reason why this can't exist. But it will require us to think differently about how we define the database. We’ll need a more holistic view from the client libraries and the admin UI to the transactional engine and the search engine. It will require a team that is emphatic to the needs of non-developers and developers and at the same time be able to dive deep into the complexities of distributed systems.

To give an example, Xata is deeply integrated into the development flow. This means that database branches are a native concept, just like Git has branches. You create database branches for your new features. These branches enable you and everyone in your team to preview, test, and collaborate. When you are ready to merge, the Xata UI guides you through the migrations needed in such a way that your application’s performance and uptime are not affected. This git-based deploy workflow is popularized by platforms like Netlify and Vercel and it is time for databases to join in.

When designing such a feature, we want to make it so simple that non-developers can be successful with it. We are aiming to expose No-Code users to developer concepts like branches and pull requests with a soft touch, so they can reach their goals today and at the same time be prepared for their application to grow tomorrow.

Today we're super excited to announce that we have raised $5M led by **Index Ventures**, with participation from **Operator Collective**, **X-Factor**, and **Firstminute**, alongside well-known tech founders, **Shay Banon and Uri Boness from Elastic**, **Neha Narkhede from Confluent**, **Guillermo Rauch from Vercel**, **Elad Gil from Color Genomics** and **Christian Bach and Mathias Biilmann from Netlify** among others.

If you'd like to be part of the team that makes this happen, please don't hesitate to reach out. We'd love to speak to you!

If you haven't heard of us before, it's because we haven't launched yet. We're busy building the product, but we don't believe in stealth mode, so we'd rather talk to you about what we're building and why.