Overview

This month has seen some very significant improvements to the brighterscript type tracking initiative, and a complete overhaul of the internals of the debug session logic, as well as many other smaller changes to bslint, the formatter, and the vscode extension. Many thanks to all who have worked so hard and contributed this month!

We need your help

The RokuCommunity projects are maintained by a relatively small group of developers (mostly volunteers), and we have a growing list of of unresolved issues. We need your help! There are many different ways you can contribute. Whether it’s addressing bugs, improving documentation, introducing new features, or simply helping us manage our expanding list of GitHub issues, your involvement would be greatly appreciated. We are more than happy to guide you in finding the most suitable contribution method that aligns with your interests. To learn more about how you can contribute, feel free to reach out to us on Slack, or explore the existing GitHub issues:

Issue of the month

In this section, we highlight a specific issue where we could benefit from the community’s assistance in finding a solution. These problems are generally straightforward to address, and serve as an excellent opportunity to become acquainted with the RokuCommunity codebases. Thanks to @iBicha (Brahim Hadriche) for the awesome job solving the June 2023 issue of the month!

This month, we’d like to draw attention to vscode-brightscript-language#470. BrighterScript has support for regular expressions, so we added some syntax highlighting to help make those stand out more. However, there’s a bug in the syntax logic, and it’s incorrectly treating valid math expressions like they’re regex expressions.

We’d like to get that fixed so math looks right again!

If you’re interested in working on this feature, please comment on the github issue or reach out to us on Slack

Editor

Align device picker name with name from the devices tab

We standardized the device names in the device picker to match the names from the devices tree view panel. We now use the user-defined name when available.

Before:

After:

Enhance host picker during launch

We’ve significantly improved the experience interacting with the device picker dropdown that is shown when using the "${promptForHost}" option when launching a debug session:

add “last used” separator for the last used device
add “devices/other devices” separator for devices not last used
add separator line to separate the “enter manually” option
refresh the list any time a new device is discovered, instead of a blanket “wait 5 seconds” that was there previously
show a loader when we think there are more devices to be discovered

Here’s a preview of the feature:

We’ve started dreaming up some fairly significant enhancements to the concept of “active devices”. You can read about them here. We’d appreciate any feedback or insight you might have into the design of the feature.

Add link for ECP registry

A new “View Registry” entry has been added to the devices panel that will open the dev channel’s registry in a web browser (if your device supports it).

Next month we’ll be adding the ability to pick what channel you want to view the registry for (instead of always defaulting to the dev channel), so stay tuned.

Debugging

Enable remote control on launch

A new launch setting called remoteControlMode has been added. This will allow you to automate the remote control mode. You can configure activateOnSessionStart and deactivateOnSessionEnd independently to fully customize how you want the feature to work. Check it out!

We’d like to give kudos to @iBicha (Brahim Hadriche) for the awesome work on this feature, as it was highlighted in our June 2023 edition as the Issue of the month.

remoteControlMode

Fix automation view crash when no config found

We fixed a bug in the automation view panel where it would crash if there was no config defined. Users could easily encounter this bug if they had never set any custom automation values.

Before the fix:

After the fix: …it just works. 😀

Fix that pesky “local variables” spinner in telnet

A common issue lately with the telnet debugger is that it gets in these strange “stuck” states, where the local variables panel seems to load infinitely. Like this:

local-spin

The telnet debugger is extremely brittle. This is because we essentially wrote a regular expression output log scraping state machine. Roku changing even a single character of their telnet output logs puts us at risk of falling into an unstable state.

We’ve fixed a few issues related to this, so the telnet debugger should be a bit more stable now. Please raise an issue if you encounter any more of these unstable situations.

Debug Protocol Enhancements

We’ve finally landed a significant refactor of our debugger after working on it for over a year (it wasn’t nonstop, but still took a while…).

Code that interacts with a Roku device directly is typically very difficult to unit test. To mitigate that, we created a debug protocol server which emulates the server-side of Roku’s binary debug protocol. This vastly simplifies the unit test experience, as we can now control both the client side and the server side, mocking both if necessary.

The one caveat is that these are not on-device tests, but the majority of our logic can be tested in this fashion. This DebugProtocolServer can also be used to provide actual debugging experiences in a brs emulator (if someone ever gets around to implementing that…).

Each ProtocolRequest and ProtocolResponse class will support loading from json, loading from a buffer, and serializing to a buffer. This makes the binary structures for client and server first-class citizens of the project, and simplifies the unit test experience as well. For the average user, this should result in a much more consistent debugging experience, and less frequent breaking changes as we release updates.

You can check out the roku-debug#107 if you’re interested in reviewing the 17,000 lines of added/changed/removed code.

Better compile error handling

Our compile error handling broke sometime in the last year or two. We used to highlight the line that had the error, but lately it just silently fails and only sometimes shows the error in the output logs.

This month we released some significant improvements to that experience. Now when a debug session detects a compile error, we will highlight the failed line in a similar style to runtime exceptions. We also keep the debug session alive, so you will need to click “stop” to terminate the session. This more closely aligns us with other debuggers like Node and dotnet. This works for both telnet and the debug protocol.

compile-error

Better debug session process cleanup

Some of our users have been experiencing some high CPU usage after a debug session has ended. This seems to be caused by the roku-debug process not properly shutting down. This month we squashed several bugs related to that. We do a much better job cleaning up opened sockets, event listeners, telnet connections, etc.

There’s not much to show here, but hopefully we found all the areas that were blocking the process shutdown. Please raise an issue if you encounter any more of these unstable situations.

Formatting

new `color-format` bslint rule to enforce specific color formats

We’ve added several new bslint rules to enforce specific color formats in your code. Here are some of the details:

color-format: ensures that all the color values follow the same prefix formatting. Can also use to prevent any colors values from being defined in the code-base (brs or bs files), except for values in a stand-alone file (ie. theme file).
- hash-hex: enforces all color values are type string or template string and use a # prefix
- quoted-numeric-hex: enforces all color values are type string or template string and use a 0x prefix
- never: enforces that no color values can be defined in the code-base (brs or bs files). Useful if you define colors in a separate stand-alone file. To use this option you would list your stand-alone file in the ignore list or diagnosticFilters.
- off: do not validate (default)

You can also specify some other settings such as color-case to enforce upper-case or lower-case characters, color-alpha to require alpha to be defined (or not defined), and color-cert to enforce Roku’s broadcast safe color 6.4 certification requirement

BrighterScript

BrighterScript Preview features

This month we’re giving the brighterscript v0.66 branch its own entire section again, because there has been a LOT of significant progress made. Almost all of this work has been implemented by @markwpearce (Mark Pearce), who is doing an incredible job moving us closer to the finish line.

You can try most of these out by installing the latest v0.66 alphas:

npm install brighterscript@next

File api

At long last, the file api has been merged! (into the upcoming v0.66 release branch). We had hoped to land this in mainline back in the spring, but there were a few breaking changes that we had to work through. Since v0.66 will bring several other breaking changes, it made more sense to go ahead and group the file API changes in with that release. We won’t get into too much detail on the file api, but here are some highlights:

all files matched by the files array are now loaded into the program. Known files (like xml and brs/bs will be handled by BrighterScript, and all other files will be passed through as-is.
plugins can now intercept the provideFile plugin hook and contribute their own enriched files (like json validation, non-scenegraph xml support, etc).

You can review pull request at brighterscript#408, or read the file api docs here

Also note that this feature is still in preview. We may still need to make some changes to the API as developers start to more heavily leverage the file API, so just keep that in mind.

Support overloaded methods in BrightScript interfaces

In order to keep the native scenegraph/brightscript data up-to-date, we regularly maintain a JSON file of all the various built-in types provided by the Roku platform. We have added support for BrightScript interface method overloads so that there is now a single definition of the method that can be added to the type system. We also fixed some incorrect function signatures like array.sort() flags param being optional.

Adds `callFunc` as member method to Custom Components

The type system now supports adding callfunc members to custom components. This will allow us to detect when an unknown function is callfunc’d.

Semantic Tokes for Native Components and Type completion in Type Expressions

We added support for better semantic token highlighting for component types and interfaces:

Code completion in Type Cast/Type Expression:

We’ve updated the code completions to now show all of the native interfaces.

Fixes compatibility of built in types (roArray -> typed arrays)

brighterscript#925 fixes some bugs in the type tracking related to assignability between string and roString and also when comparing typed arrays

Fixes enum validation

Fixed a bug in enum validation where returning an integer-based enum value should not cause an error, as the value should be convertible. We also fixed a few issues where enums were not being properly treated as their underlying types when passed to functions.

Allow classes and native components in Typed Arrays

We fixed a bug When you declare a type of native component array, (eg. roAssociativeArray[] or roBitmap[]) there is a validation error. The following issue should no longer be a problem:

Improve type compatibility diagnostic messages

We’ve improved some of the “type a is not compatible with type b” messages. One change is that we now truncate the list of items with a ...and x more message.

Similar to typescript, this will show exclusively the “missing members” message if there are any missing members. Then, if there are no missing members, it’ll show the “members are incompatible” message if there are any incompatible members. This helps reduce the size of these large errors in some codebases. You’ve got to address the missing members and the incompatible types, so it doesn’t hurt to just show one at a time.

combined error message before:

Missing members after:

Incompatible members after:

Better multi scope messages

We improved the multi-scope diagnostic messaging for related infos more relevant (and concise). Also truncate that list since seeing 95 related infos is just noise…

Here’s how it looks now:

Fixed some bugs in hovers

Adds more incomplete expressions to the AST, so they can be used for completions.

Fixes:

Union type completions are only symbols available in all types Completion on union type should only show shared properties
Array toString() uses default type Reduce like types for hovers
Fixes Enum completions and hovers Completions for enum is wrong

You can read more about it here: brighterscript#874

Wider support for `as object`

We’ve fixed some type validation issues related to as object variable types. They are permitted to be passed to any function type (just like dynamic), so we’ve fixed a bug that was marking those as errors.

Various other type validation enhancements

BinaryExpressions and UnaryExpressions should be able to infer resultant types (eg. Integer * Integer => Integer, String + String => String, etc.)
Change Hover & Completion to use SymbolTable instead of variableDeclarations/callables

This will complete the refactor to move away from using these lookup tables to unify how a symbol can be discovered/known. When this is done, the concept of VariableDeclarations can be removed from the code.

Validate function calls for argument type validity

Function argument equality

sub main()
    printMessage(1)
    '            ~ Argument of type 'number' is not assignable to parameter of type 'string'
end sub

function printMessage(message as string)
    print message
end function

Better diagnostic ranges for unknown dotted get expressions

We’ve tightened the range for the “can’t find this property” diagnostics so it only highlights the property instead of the entire starting expression.

Before:

After:

Adds .kind prop to AstNode.

The is<SomeAstNodeType> functions from reflections.ts are used heavily across brighterscript. As such, we found that this expression thing?.constructor?.name === 'Whatever' was fairly slow. To speed it up, we added a .kind property to AstNode, an AstNodeKind enum., and then converted the reflection methods to check for .kind instead of the thing?.constructor?.name === 'Whatever' logic. Raw benchmarks show this as a fairly significant boost:

What that means in practice is validation speeds improve by 10-12 percent.

Downsides:

This is a breaking change, as all plugins that produce AST would need to be upgraded to the latest version of brighterscript in order to work with the version of brighterscript shipped with the vscode extension. Or, for example, latest version of brighterscript being used with a plugin that depends on an older version of brighterscript.

Breaking changes

We’ve introduced some additional breaking changes to brighterscript as we move closer to the release of v0.66.0.

all plugin event handler parameters have been converted single event objects. (brighterscript#824)
several plugin event names have changed as a result of the file api. We’ve added warnings if your plugins use them, but plugins will need to update to the new names.
the XML AST has been refactored to better support being modified by plugins during transpile. (brighterscript#818)
the .kind property on AstNode has been added, and all the reflection methods exclusively check for that now (#799)

Thank you

Last but certainly not least, a big Thank You to the following people who contributed this month:

Contributions to vscode-brightscript-language:

@fumer-fubotv (fumer-fubotv)
- Add ability to capture device screenshots (PR #505)
@iBicha (Brahim Hadriche)
- Enable remote control on launch (PR #503)
@MilapNaik (Milap Naik)
- Add link for ECP registry (PR #511)
@TwitchBronBron (Bronley Plumb)
- Fix error when generating the docs (12dfb27)
- Fix automation view crash when no config found (PR #504)
- Diagnostic manager (PR #502)
- Make device picker name same as tree view item (PR #508)
- Add brs to releases script (5c6622b)
- Enhance host picker during launch (PR #512)

Contributions to brighterscript:

@markwpearce (Mark Pearce)
- Allow classes and native components in Typed Arrays (PR #919)
- Fixes some enum validation stuff (PR #920)
- Fixes compatibility of built in types (roArray -> typed arrays) (PR #925)
- Semantic Tokes for Native Components and Type completion in Type Expressions (PR #927)
- Adds callFunc as member method to Custom Components (PR #929)
- Allows scrape-roku-docs to consolidate overloaded methods (PR #930)
@TwitchBronBron (Bronley Plumb)
- Better multi scope messages (PR #904)
- Refine type compat message (PR #908)
- Better typing for Deferred (PR #923)
- Add interface parameter support (PR #924)
- ci: Don’t run test-related-projects on release since it already ran on build (157fc2e)
- File api (PR #408)
- Fix stagingDir issue in transpiler test (8a31693)

Contributions to roku-debug:

@TwitchBronBron (Bronley Plumb)
- Fix bug with telnet getting stuck (PR #163)
- Debug Protocol Enhancements (PR #107)
- Clean up control socket when it’s closed (PR #166)

Contributions to bslint:

@disc7 (Charlie Abbott)
- [WIP] Color format checking for bslint (PR #94)
@TwitchBronBron (Bronley Plumb)
- copy expectDiagnostics from brighterscript for tests (PR #95)
- Fixing issues before release 0.8.11 (051fd4c)