Cats Effect (JS Only) tests over WASMGC

by Manav

6 min read

Cats Effect Scala.js Scala Wasm Wasm tests

building up to running the Cats Effect test suite on Wasm(GC).

current state of WASM(GC) over Scala.js is experimental. @tanishiking is working on this proposal, to Add Support for Wasm Component Model and WASIp2, which means that Not all tests in cats-effect can run over WASM due to JavaScript-specific dependencies and runtime assumptions. To make the tests work over WASM, we would need to refactor the code, provide WASM-specific implementations, and exclude incompatible tests, but our current goal is to atleast make sure that cats-effect tests can run over WASM or NOT.

Setting up the Environment

we have already done the preliminary step — which was to make sure that Cats Effect can run with Wasm. TO setup the environment, i am going with cloning the Typelevel/cats-effect repository, and work on that to make sure it runs the prebuilt cat-effect JS only tests. I want to change as Minimal to the codebase such that it comes in a situation that it just Works (not more than that). Since we want to check if wasm can build for existing tests in cats-effect we will make sure not to change any test files in the directory.

currently JS tests exists in tests/js/src/ of the root directory with file structure:

  > : tree tests/js/src/
tests/js/src/
├── main
│   └── scala
│       ├── cats
│       │   └── effect
│       │       └── DetectPlatform.scala
│       └── catseffect
│           └── examplesplatform.scala
└── test
    └── scala
        └── cats
            └── effect
                ├── exports.scala
                ├── IOPlatformSuite.scala
                ├── RunnersPlatform.scala
                ├── std
                │   └── ConsoleJSSuite.scala
                ├── SyncIOPlatformSuite.scala
                └── unsafe
                    ├── BatchingMacrotaskExecutorSuite.scala
                    ├── JSArrayQueueSuite.scala
                    ├── SchedulerSuite.scala
                    └── StripedHashtableSuite.scala

Changes to CODE

Let's start by creating separate lazy values for both wasmSettings, and testJS which we will eventually use to partially run tests over only the JsSuite.

lazy val wasmSettings = Seq(
scalaJSLinkerConfig := {
  scalaJSLinkerConfig
    .value
    .withExperimentalUseWebAssembly(true)
    .withModuleKind(ModuleKind.ESModule)
},
jsEnv := {
  import org.scalajs.jsenv.nodejs.NodeJSEnv
  val config = NodeJSEnv
    .Config()
    .withArgs(
      List(
        "--experimental-wasm-exnref",
        "--experimental-wasm-imported-strings",
        "--turboshaft-wasm"
      ))
  new NodeJSEnv(config)
}
)

lazy val testsJS = tests
.js
.settings(wasmSettings)
.settings(
  libraryDependencies ++= Seq(
    "org.scala-js" %%% "scala-js-macrotask-executor" % "1.1.1" % Test,
    "org.scalatest" %%% "scalatest" % "3.2.18" % Test,
    "co.fs2" %%% "fs2-io" % "3.9.4" % Test
  ),
  Test / scalaJSUseMainModuleInitializer := false,
  Test / fork := false
)

also, lets add Scala.js repository as resolver for just in case, and updating the lazy val for rootJS.

resolvers += "Scala.js Repository" at "https://repo.scala-js.org/releases"
lazy val rootJS =
  project.aggregate(jsProjects: _*).aggregate(testsJS).enablePlugins(NoPublishPlugin)

that's it? right but there's a problem that Argument +l becomes unrecognised by ScalaTest's Runner, assuming that there's no use of this for our testsJS lazy val.

ok! let's remove

ThisBuild / Test / testOptions += Tests.Argument("+l")

But now we face yet another error, the undefined value issue (java.lang.ClassCastException) which says org.scalajs.linker.runtime.UndefinedBehaviorError: java.lang.ClassCastException: undefined cannot be cast to java.lang.String, for which we need to move the JavaScript-specific code (e.g., js.Dynamic, js.isUndefined) into the actual Scala.js test files, not in the build.sbt file. Since js object and related functionality are only available in Scala.js source files. They cannot be used in the build.sbt file. And in the test file, we can use js.isUndefined to check for undefined values and handle them appropriately.

THE FIX

is to change current implementation of tests/js/src/test/scala/cats/effect/std/ConsoleJSSuite.scala by

package cats.effect.std

import cats.effect.IO
import cats.effect.unsafe.implicits.global
import org.scalatest.funsuite.AsyncFunSuite
import fs2.io.stdin

class ConsoleJSSuite extends AsyncFunSuite {
  test("Console should write to stdout") {
    val program = IO.println("Hello, world!")
    program.unsafeRunAndForget()
    succeed
  }

  test("Console should write to stderr") {
    val program = IO.println("Error occurred!")
    program.unsafeRunAndForget()
    succeed
  }

  test("Console should read from stdin") {
    val program = stdin[IO](1024).compile.drain // Replace IO.readLine with fs2.io.stdin
    program.unsafeRunAndForget()
    succeed
  }
}

Also, the JavaScript environment may not treat the code as an ES module, leading to runtime errors, for which lets add a type 'module' in package.json.

{ "type": "module" }

Looks like we're done

lets add libraryDependencies for scalajs-env-nodejs and scalajs-linker-interface.

Adding another test option in our lazy val for testJS, to exclude unsupported tests.

    Test / testOptions := Seq(
      Tests.Filter(test => !test.contains("UnsupportedSuite"))
    ),

boom lets try this

  > : sbt clean
∙ sbt update
∙ sbt testsJS/test

Output

I have pasted the output on paste.rs and paste.rs (with Unsupported test filter) for your reference, Yes it passes some of the JS tests but i have no idea why it has some tests sucked in time:

  | => cats.effect.IOSuite 5067s
  | => cats.effect.IOParSuite 5067s
  | => cats.effect.std.DispatcherSuite 5067s
  | => cats.effect.std.MutexSuite 5068s
  | => cats.effect.kernel.RefSuite 5068s
  | => cats.effect.IOMtlLocalSuite 5068s
  | => cats.effect.IorTIOSuite 5068s
  | => cats.effect.std.UnboundedQueueSuite 5068s

these tests are suck in sbt testsJS/test loop, which means there are some missing tests in output like

  • cats.effect.unsafe.JSArrayQueueSuite
  • cats.effect.unsafe.IORuntimeSuite
  • cats.effect.unsafe.IORuntimeConfigSuite
  • cats.effect.std.BoundedQueueSuite
  • cats.effect.std.CountDownLatchSuite
  • cats.effect.SyncIOSuite

But let's see the positive side that it passes some tests over WASM(and we can clearly see them in output).

Passed Tests from the Output:

Note

(includes output of lazy val with Unsupported Filter)

  • cats.effect.unsafe.IORuntimeBuilderSuite:

    • cleanup allRuntimes collection on shutdown
    • configure the failure reporter

  • cats.effect.ExitCodeSuite:

    • ExitCode.unapply is exhaustive

  • cats.effect.std.internal.BankersQueueSuite:

    • maintain size invariants
    • dequeue in order from front
    • dequeue in order from back
    • reverse

  • cats.effect.kernel.ParallelFSuite:

    • ParallelF[PureConc]: parallel.isomorphic functor

    • ParallelF[PureConc]: parallel.isomorphic pure

    • ParallelF[PureConc]: parallel.parallel round trip

    • ParallelF[PureConc]: parallel.sequential round trip

    • ParallelF[PureConc]: commutative applicative.ap consistent with product + map

    • ParallelF[PureConc]: commutative applicative.applicative homomorphism

    • ParallelF[PureConc]: commutative applicative.applicative identity

    • ParallelF[PureConc]: commutative applicative.applicative interchange

    • ParallelF[PureConc]: commutative applicative.applicative map

    • ParallelF[PureConc]: commutative applicative.applicative unit

    • ParallelF[PureConc]: commutative applicative.apply commutativity

    • ParallelF[PureConc]: commutative applicative.apply composition

    • ParallelF[PureConc]: commutative applicative.covariant composition

    • ParallelF[PureConc]: commutative applicative.covariant identity

    • ParallelF[PureConc]: commutative applicative.invariant composition

    • ParallelF[PureConc]: commutative applicative.invariant identity

    • ParallelF[PureConc]: commutative applicative.map2/map2Eval consistency

    • ParallelF[PureConc]: commutative applicative.map2/product-map consistency

    • ParallelF[PureConc]: commutative applicative.mapOrKeepToMapEquivalence

    • ParallelF[PureConc]: commutative applicative.monoidal left identity

    • ParallelF[PureConc]: commutative applicative.monoidal right identity

    • ParallelF[PureConc]: commutative applicative.productL consistent map2

    • ParallelF[PureConc]: commutative applicative.productR consistent map2

    • ParallelF[PureConc]: commutative applicative.replicateA_ consistent with replicateA.void

    • ParallelF[PureConc]: commutative applicative.semigroupal associativity

    • ParallelF[PureConc]: align.align associativity

    • ParallelF[PureConc]: align.align homomorphism

    • ParallelF[PureConc]: align.alignWith consistent

  • cats.effect.std.internal.BinomialHeapSuite:

    • dequeue by priority
    • maintain the heap property
    • maintain correct subtree ranks

  • and others:

    • cats.effect.unsafe.JSArrayQueueSuite
    • cats.effect.unsafe.IORuntimeSuite
    • cats.effect.std.UnsafeBoundedSuite
    • cats.effect.unsafe.IORuntimeConfigSuite
    • cats.effect.std.BoundedQueueSuite
    • cats.effect.std.CountDownLatchSuite
    • cats.effect.SyncIOSuite
    • cats.effect.CallbackStackSuite
    • cats.effect.std.internal.BinomialHeapSuite

Conclusion

Output seems to focus more on the failed tests due to the java.lang.ClassCastException. The tests that are passing are primarily those that do not involve complex type casting or interactions with JavaScript/Wasm-specific runtime behavior. The failing tests are consistently failing due to a ClassCastException, which suggests that there may be an issue with how certain types are being handled in the Scala.js/Wasm environment.

To resolve these issues, we may need to investigate the specific parts of the code where the ClassCastException is being thrown, particularly in the context of how Scala.js handles type casting and interoperability with JavaScript/Wasm.

Reference