Using TypeSense

Client

The client is fully set up and shouldn't require further configuration. This part of the code ensures we have a connection to Typesense and that the search functionality works.

Services

TypesenseInitializationService

This is where all the initial setup happens. After we have a client, we want to ensure all our data from the database (the ones we want to search on) are available in Typesense. This service handles that. Generally, you shouldn't need to change much here except in two places: the migrate and import functions. More on how to set this up later.

TypesenseCollectionService

Collections are containers that hold groups of documents with similar structures. Each collection is like a database table where all the documents share the same set of fields.

TypesenseDocumentService

In Typesense, documents are individual units of data that you store and search through. Each document represents a single item (like a product, article, or user profile) that can be indexed and retrieved in search results. This is also fully set up and ready to use.

TypesenseQueryService

Lastly, we have the Query service. This service handles all the search functionality. If you want to search one collection for a record or search the entire Typesense "database," this is your go-to service.

Controller

There is also a Typesense controller. This is mainly used to migrate and import Typesense. But what does that mean? You use migrate if you have made changes to the way a document or collection is stored in Typesense. For example, if you changed a column's name or type, you need to migrate before these changes take effect. Be aware that a migrate removes all data from Typesense! After a migrate, you need to import to ensure all your data in the database is also available in Typesense.

Now that we know most of how Typesense integrates into our project, it's time to add a new table to Typesense. But how do we do that?! 🤔

Adding the good stuff

It's easy! We have built Typesense in such a way that you need to do minimal work to get up and running with a new table.

In src/modules/typesense/enums/typesense-collection-index.enum.ts, add the name of your new collection. For the sake of this example, let's use Employee.

export enum TypesenseCollectionName {
  USER = 'user',
  EMPLOYEE = 'employee'
}

export interface MultiSearchResult {
  [TypesenseCollectionName.USER]?: PaginatedOffsetResponse<UserSearchSchema>,
  [TypesenseCollectionName.EMPLOYEE]?: PaginatedOffsetResponse<EmployeeSearchSchema>
}

As you see, we use a file named EmployeeSearchSchema. But this doesn't exist yet.

In src/modules/typesense/collections/, add a new SearchCollection. This is where we allow the code to search on specific values in Typesense. Add a file named employee.collections.ts.

export interface EmployeeSearchSchema {
  id: string
  uuid: string
  firstName: string
  lastName: string
  role: Role
}

export class EmployeeTypesenseCollection extends TypesenseCollection {
  readonly name = TypesenseCollectionName.EMPLOYEE

  readonly searchableFields = [
    { name: 'firstName', type: 'string', sort: true },
    { name: 'lastName', type: 'string', sort: true }
  ] as const

  readonly filterableFields = [
    { name: 'role', type: 'string', optional: true }
  ] as const
}

The EmployeeSearchSchema defines how the data sits in Typesense. Think of it as your entity in the database but only with the parts you need to search on.

The searchableFields are the values we can search on. It defaults to sorting the output of the value you're searching for. Note that this is also fuzzy search, meaning you can type things incorrectly, and it will still find it. Pretty neat, huh? 😄

The filterableFields are the parts we filter with. Does your Employee need to have a specific Role? You can make this optional or not.

How the filtering and searching work will be discussed later on.

There are a few more things we need to do.

How do we get our data into Typesense in the first place? We know it has to do with the collectionService and the documentService. But how does our application know what to add?

Well, glad you asked!

In src/modules/typesense/services/collectors, create a employee-typesense.collector.ts file.

@Injectable()
export class EmployeeTypesenseCollector implements TypesenseCollector {
  constructor (
    private readonly employeeRepository: EmployeeRepository
  ) {}

  transform (employees: Employee[]): EmployeeSearchTransformerType[] {
    return new EmployeeSearchTransformer().array(employees)
  }

  async fetch (uuids?: string[]): Promise<Employee[]> {
    return await this.employeeRepository.find({
      where: { uuid: InOrIgnore(uuids) },
      relations: { role: true }
    })
  }
}

The fetch function gets all the information we need to put into Typesense. This find function finds all the records with the necessary relations and possible UUIDs if needed.

The transform function transforms the object we get from Typesense to our object in the format we need.

We will add that in a few moments, but first, make sure our Typesense can access this collector.

Head over to src/modules/typesense/services/collectors/typesense-collector.factory.ts and add:

private readonly employeeTypeSenseCollector: EmployeeTypesenseCollector

In the constructor:

case TypesenseCollectionName.EMPLOYEE:
      return this.employeeTypeSenseCollector

Add this part in the Create switch case.

For more info about this pattern click here

Now all we have to do is create the transform.

In src/modules/typesense/transformers, create a new file employee.transformer.ts.

export class EmployeeSearchTransformerType {
  id: string
  uuid: string
  firstName: string
  lastName: string
  role: Role
}

export class EmployeeSearchTransformer
  extends Transformer<Employee, EmployeeSearchTransformerType> {
  transform (employee: Employee): EmployeeSearchTransformerType {
    return {
      id: employee.uuid,
      uuid: employee.uuid,
      firstName: employee.firstName ?? '',
      lastName: employee.lastName ?? '',
      role: employee.role,
    }
  }
}

Remember I told you to add two things in the initializationService?

Now it's time to add those two things.

In the import function, add the following:

if (indexes.includes(TypesenseCollectionName.EMPLOYEE)) {
  await this.typesenseCollectionService
    .importToTypesense(TypesenseCollectionName.EMPLOYEE)
}

And in the migrate function:

if (indexes.includes(TypesenseCollectionName.EMPLOYEE)) {
  await this.migrateCollection(
    TypesenseCollectionName.EMPLOYEE,
    new EmployeeTypesenseCollection().getSchema(),
    fresh
  )
}

Congrats! You have set up Typesense for a new entity! 🎉

Links

Typesense Documentation
Factory Pattern
Factory Pattern - Typescript