Contributing a fix to playwright-ruby-client

tl;dr

It was nice to be able to contribute (I think a fairly meaningful) fix to an open source project, YusekeIwaki/playwright-ruby-client.

The issue: If the behavior you're asserting is not happening is initially happening, the test crashes.

• Specifically, a negative assertion via native RSpec negation, e.g. expect(modal).not_to be_visible.

Expected behavior: The assertion should wait until the assertion is satisfied, otherwise time out.

The technical details of the fix are in the PR.

Context

At work, we're transitioning from Capybara RSpec matchers to playwright-ruby-client for our system specs. It enables us to use assertions that we're already familiar with from our React unit tests on the Ruby side of things + browser assertion API improvements.

A coworker of mine asked to pair on a test they were having issues getting to pass (not the exact test, same core concept though):

it 'hides modal after creating Thing' do
  page.goto('/things')
  page.get_by_role('button', name: 'Add Thing').click
 
  modal = page.get_by_role('dialog')
  expect(modal).to be_visible
 
  modal.get_by_role('textbox', name: 'Name')
  modal.get_by_role('button', name: 'Submit').click
 
  expect(modal).not_to be_visible
end

• Frontend worked as expected
• We figured out that expect(modal).not_to be_visible was the source of the issue (see the PR for more details)
• Quick fix: change matcher to expect(modal).to not_be_visible
  • notably, this matcher is defined by playwright-ruby-client and is not making use of native RSpec negation

We were unblocked, but I wasn't satisfied with leaving that bug in playwright-ruby-client.

I figured it would be a good opportunity to contribute something back to this library that powers our system specs.

So, I investigated it on my own time.

Investigation

It turned out, all the matchers that poll until the expectation is asserted successfully (this describes almost every matcher) had a bug when using RSpec native negation, i.e. expecting not_to vs. to.

Using the example above:

• expect(modal).not_to be_visible would be executed when the modal was visible and crash the test
• expected behavior is waiting until the modal was no longer visible until the test times out, rather than crashing

And I found this behavior to be present for all eventual matchers: if the thing you're asserting is not happening is initially happening, the test crashes.

I put up the fix. It was reviewed and merged within the week. It was nice to have an opportunity to give back.