Deploy Without Fear
Using Automated Tests

Chris Stone
https://chrisstone.dev

QR code for talk notes and slides

My goals today

  • Share lessons learned adding automated testing
  • Teach some basics of types of tests
  • Show my appreciation of automated tests and what they can do for us
  • Inspire people here to write some great tests

Why don't people write tests?

  • Not sure what to test
  • Not sure what a "good test" looks like
  • Someone else is in charge of the tests
  • Content to test manually

Why do I write tests?

  • I get scared 😭
  • I see code I don't understand
  • I'm worried the deployment had problems
  • I'm sure I'll forget to test something manually
I don't know what any of this is and I'm scared

Dealing with fear

I must not fear. Fear is the mind-killer. Fear is the little-death that brings total obliteration. I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain.
Dune by Frank Herbert
Dune by Frank Herbert

Dealing with fear

Write tests until fear is transformed into boredom.
Test-Driven Development by Example by Kent Beck
Test-Driven Development
By Example

by Kent Beck

Dealing with fear

Write tests until fear is transformed into boredom.

If something in our application scares me,
that's where I write a test.

If I don't worry about something not working,
then I don't worry about testing any further.

Why do I write tests?

  • I get scared
  • Code coverage
  • Regression detection
  • Avoid writing superfluous code
  • Executable documentation

Why do I write tests?

Code coverage

  • 100% is not realistic and not useful.
  • You can increase coverage while not testing things of value.

Why do I write tests?

Regression detection

  • Tests should run on every commit in CI/CD
  • A good test is possible to fail someday
  • Address flaky tests, team should have faith in the test suite
    • Fix if you can
    • Skip until you get the time, if necessary

Why do I write tests?

Avoid writing superfluous code

  • Test-driven development (TDD)
  • Red-green-refactor cycle will help drive simple design to avoid accidental complexity
    • Remember, refactoring preserves behavior!

Why do I write tests?

Tests are the best documentation

Confluence screenshot
Extensive documentation outside the code

Why do I write tests?

Tests are the best documentation


export function isLevelInDanger(oldScores: number[],
		newScore: number) {
	// An alert should be displayed
	// if the new score is less than average.
	const sum = oldScores.reduce((a, b) => a + b);
	return newScore < (sum / oldScores.length);
}
  	

Comments can lie

Why do I write tests?

Tests are the best documentation


export function isLevelInDanger(oldScores: number[],
		newScore: number) {
	// An alert should be displayed
	// if the new score is less than average.
	const sum = oldScores.reduce((a, b) => a + b);
	return newScore <= (sum / oldScores.length);
}
  	

Comments can lie

Why do I write tests?

Tests are the best documentation


export function isLevelInDanger(oldScores: number[],
		newScore: number) {
	const sum = oldScores.reduce((a, b) => a + b);
	return newScore <= (sum / oldScores.length);
}
  	

it("should only alert when less than average", () => {
	const actual = isLevelInDanger([1, 2, 3], 2);
	assert.deepStrictEqual(actual, false); // FAIL
});
  	

A test will fail if it becomes untrue.

Why do I write tests?

Tests are the best documentation

What does it mean for writing to be effective?

  • Readable
  • Relevant
  • Applicable

Types of tests

  • Unit
    • Tests a small unit of isolated simple design code
    • Mocks out any calls to an external dependency, data layer, or API interface
    • Runs in milliseconds
  • Integration
    • A relatively small area of code that is allowed to talk to other components or a test database interface
    • Runs up to a few seconds
  • End-to-end
    • Tests a full user behavior journey using browser protocols or emulated devices
    • Runs up to tens of seconds

Types of tests

Which type to pick

Test Pyramid
Test Pyramid
Test Pyramid
Test Trophy

Pick the lowest level that gives value.

Which type to pick


export function isLevelInDanger(oldScores: number[],
		newScore: number) {
	const sum = oldScores.reduce((a, b) => a + b);
	return newScore <= (sum / oldScores.length);
}
  
Unit Integration End-to-end
Effort
Value

Which type to pick


var dbStaticResources []db.StaticResource
query := s.dbClient.
    Model(db.StaticResource{}).
    WithContext(ctx).
    Distinct().
    Select(
       "static_resources.id",
       "static_resources.grade",
       "static_resources.title",
    )

if input != nil {
    if input.ResourceType.IsValid() {
       query = query.Where("static_resource_type = ?", input.ResourceType)
    }
    if len(input.Grades) > 0 {
       query = query.Where("grade IN (?)", input.Grades)
    }
    if len(input.SubjectAreas) > 0 {
       query = query.Joins(`INNER JOIN subject_area_resource_mappings ON
             subject_area_resource_mappings.static_resource_id=static_resources.id             AND subject_area_resource_mappings.deleted_at IS NULL`).
          Where("subject_area_resource_mappings.subject_area_id IN (?)", input.SubjectAreas)
    }
    if len(input.TaskTypes) > 0 {
       query = query.Joins(`INNER JOIN task_type_resource_mappings ON
             task_type_resource_mappings.static_resource_id=static_resources.id             AND task_type_resource_mappings.deleted_at IS NULL`).
          Where("task_type_resource_mappings.task_type_id IN (?)", input.TaskTypes)
    }
    if len(input.Keywords) > 0 {
       query = query.Joins(`LEFT JOIN keywords_resource_mappings ON
             keywords_resource_mappings.static_resource_id=static_resources.id             AND keywords_resource_mappings.deleted_at IS NULL`).
          Joins(`LEFT JOIN keywords ON keywords.id=keywords_resource_mappings.keyword_id
             AND keywords.deleted_at IS NULL`)

       innerQuery := s.dbClient.WithContext(ctx)
       for index := range input.Keywords {
          innerQuery = innerQuery.Or("keywords.keyword ~* ? OR static_resources.title ~* ?",
             fmt.Sprintf(`.*%s.*`, input.Keywords[index]),
             fmt.Sprintf(`.*%s.*`, input.Keywords[index]))
       }
       query = query.Where(innerQuery)
    }
}

err := query.Joins("File", func(db *gorm.DB) *gorm.DB {
    return db.Select("id", "url", "thumbnail_url")
}).Find(&dbStaticResources, "static_resources.is_archived = false").Error
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
    return nil, fmt.Errorf("services.GetAll: Error running query find: %w", err)
}
  
Unit Integration End-to-end
Effort
Value

Which type to pick


// test that a user can record audio and submit a message
  
Unit Integration End-to-end
Effort
Value

How to get started

You have to start somewhere

Test the change you want to see in the world

Be the change you want to see in the world

Adding back-end tests

  • The more we prepare data in the backend for how the frontend will use it, the more we can rely on quick back-end unit tests.
  • Difference between a unit and integration test may just be whether an interface is mocked or if its implementation is actually used.
Testify logo

Adding back-end tests


func TestMyFunction(t *testing.T) {
	var myTests = []struct {
		inputValue     int
		expectedOutput string
	}{
		{inputValue: 2, expectedOutput: "critical"},
		{inputValue: 3, expectedOutput: "low"},
		{inputValue: 10, expectedOutput: "high"},
	}
	for _, tt := range myTests {
		actualOutput, actualError := MyFunction(tt.inputValue)
		require.Equal(t, tt.expectedOutput, actualOutput)
	}
}
		

Adding end-to-end tests

  • Highest and smallest on both the pyramid and the trophy
  • That doesn't mean to not write them; they can be very helpful
Playwright logo

Adding end-to-end tests


test('teacher can create new assignment',
		async ({ page }) => {
	await this.page.goto('/login');
	await this.page
      .locator('.MuiCircularProgress-svg')
      .waitFor({ state: 'hidden' });
	await this.page.getByRole('combobox').fill(districts.newDistrict.districtName);
	await this.page
		.getByRole('option', { name: district, exact: true })
		.click();
	await this.page.getByRole('textbox', { name: 'Username' }).fill(users.teacher.email);
	await this.page.getByRole('textbox', { name: 'Password' }).fill(process.env.USER_PASSWORD!);
	await this.page.getByRole('button', { name: 'Login' }).click();
	await expect(this.page.getByLabel('account of current user')).toContainText(
		`${users.superAdmin.firstName} ${users.superAdmin.lastName}`,
	);
	await this.page.getByRole('button', { name: 'create Assignment' }).click();
	await this.page
		.getByRole('main')
		.getByRole('button', { name: '​', exact: true })
		.click();
	await this.page.getByText(className).click();
	// ...
});
  	

We started with Playwright Codegen

Adding end-to-end tests


test('teacher can create new assignment',
		async ({ loginPage, header, teacherPage }) => {
	await loginPage.login(
		districts.newDistrict.districtName,
		users.teacher.email,
		process.env.USER_PASSWORD!,
	);
	await expect(header.usernameButton).toContainText(
		`${users.teacher.firstName} ${users.teacher.lastName}`,
	);
	await teacherPage.createNewAssignment();
	await teacherPage.chooseClass(classes.newClass.className);
	// ...
});
  

We refactored with Page Object Model

Adding end-to-end tests


test.use({ storageState: StorageStates.teacher });
test('teacher can create new assignment',
		async ({ teacherPage }) => {
	await teacherPage.page.goto('/');
	await teacherPage.createNewAssignment();
	await teacherPage.chooseClass(classes.newClass.className);
	// ...
});
  	

Took advantage of Storage State

Adding front-end tests

  • Many libraries exist that help with both unit tests and integration tests.
  • Some front-end testing may be considered "component testing".
Vite logo ➡️ Vitest logo

Adding front-end tests


const keywords: MockedResponse<GetKeywordsQuery> = {
	request: {
		query: GetKeywordsDocument,
	},
	result: {
		data: {
			keyword: [{ id: '1234', keyword: 'hello' }],
		},
	},
};
it('should add new keyword', async () => {
	const onChange = vi.fn();
	render(
		<MockedProvider mocks={[keywords]}>
			<AddKeywordTags onChange={onChange} />
		</MockedProvider>
	);

	const combobox = screen.getByRole('combobox');
	await waitFor(() => expect(combobox).not.toBeDisabled(), {
		timeout: 500,
	});
	await userEvent.type(combobox, 'world');
	expect(screen.getByDisplayValue('world')).toBeInTheDocument();
	await userEvent.keyboard('{enter}');
	expect(onChange).not.toHaveBeenCalled();
	expect(screen.getByText('Add a new keyword')).toBeVisible();
});
  

Adding to CI/CD


jobs:
  build:
    name: Lint and Test TypeScript
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Setup Node.js environment
        uses: actions/setup-node@v3
        with:
          node-version: '22'

      - name: Checkout repository under $GITHUB_WORKSPACE
        uses: actions/checkout@v3

      - name: Install dependencies
        run: |
          cd packages/apps/web
          yarn install

      - name: Lint for web-app
        run: |
          cd packages/apps/web
          yarn lint

      - name: Build for web-app
        run: |
          cd packages/apps/web
          yarn build

      - name: Test for web-app
        run: |
          cd packages/apps/web
          yarn test
		

Pull request required checks

Adding to CI/CD


jobs:
  e2e:
    name: Run the e2e tests
    uses: ./.github/workflows/_shared_run_e2e_tests.yml
    with:
      ENVIRONMENT: prod
    needs: [deploy-api, deploy-web-app]
    secrets:
      E2E_USER_PASSWORD: ${{ secrets.E2E_USER_PASSWORD }}
      SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
		

End-to-end tests after every deployment

Thank you!

Chris Stone
https://chrisstone.dev

QR code for talk notes and slides