Category Archives: Planet WebKitGTK

Igalia and WebKit: status update and plans (2024)

Published November 3, 2024 / by mario

It’s been more than 2 years since the last time I wrote something here, and in that time a lot of things happened. Among those, one of the main highlights was me moving back to Igalia‘s WebKit team, but this time I moved as part of Igalia’s support infrastructure to help with other types of tasks such as general coordination, team facilitation and project management, among other things.

On top of those things, I’ve been also presenting our work around WebKit in different venues, such as in the Embedded Open Source Summit or in the Embedded Recipes conference, for instance. Of course, that included presenting our work in the WebKit community as part of the WebKit Contributors Meeting, a small and technically focused event that happens every year, normally around the Bay Area (California). That’s often a pretty dense presentation where, over the course of 30-40 minutes, we go through all the main areas that we at Igalia contribute to in WebKit, trying to summarize our main contributions in the previous 12 months. This includes work not just from the WebKit team, but also from other ones such as our Web Platform, Compilers or Multimedia teams.

So far I did that a couple of times only, both last year on October 24rth as well as this year, just a couple of weeks ago in the latest instance of the WebKit Contributors meeting. I believe the session was interesting and informative, but unfortunately it does not get recorded so this time I thought I’d write a blog post to make it more widely accessible to people not attending that event.

This is a long read, so maybe grab a cup of your favorite beverage first…

Igalia and WebKit

So first of all, what is the relationship between Igalia and the WebKit project?

In a nutshell, we are the lead developers and the maintainers of the two Linux-based WebKit ports, known as WebKitGTK and WPE. These ports share a common baseline (e.g. GLib, GStreamer, libsoup) and also some goals (e.g. performance, security), but other than that their purpose is different, with WebKitGTK being aimed at the Linux desktop, while WPE is mainly focused on embedded devices.

This means that, while WebKitGTK is the go-to solution to embed Web content in GTK applications (e.g. GNOME Web/Epiphany, Evolution), and therefore integrates well with that graphical toolkit, WPE does not even provide a graphical toolkit since its main goal is to be able to run well on embedded devices that often don’t even have a lot of memory or processing power, or not even the usual mechanisms for I/O that we are used to in desktop computers. This is why WPE’s architecture is designed with flexibility in mind with a backends-based architecture, why it aims for using as few resources as possible, and why it tries to depend on as few libraries as possible, so you can integrate it virtually in any kind of embedded Linux platform.

Besides that port-specific work, which is what our WebKit and Multimedia teams focus a lot of their effort on, we also contribute at a different level in the port-agnostic parts of WebKit, mostly around the area of Web standards (e.g. contributing to Web specifications and to implement them) and the Javascript engine. This work is carried out by our Web Platform and Compilers team, which tirelessly contribute to the different parts of WebCore and JavaScriptCore that affect not just the WebKitGTK and WPE ports, but also the rest of them to a bigger or smaller degree.

Last but not least, we also devote a considerable amount of our time to other topics such as accessibility, performance, bug fixing, QA... and also to make sure WebKit works well on 32-bit devices, which is an important thing for a lot of WPE users out there.

Who are our users?

At Igalia we distinguish 4 main types of users of the WebKitGTK and WPE ports of WebKit:

Port users: this category would include anyone that writes a product directly against the port’s API, that is, apps such as a desktop Web browser or embedded systems that rely on a fullscreen Web view to render its Web-based content (e.g. digital signage systems).

Platform providers: in this category we would have developers that build frameworks with one of the Linux ports at its core, so that people relying on such frameworks can leverage the power of the Web without having to directly interface with the port’s API. RDK could be a good example of this use case, with WPE at the core of the so-called Thunder plugin (previously known as WPEFramework).

Web developers: of course, Web developers willing to develop and test their applications against our ports need to be considered here too, as they come with a different set of needs that need to be fulfilled, beyond rendering their Web content (e.g. using the Web Inspector).

End users: And finally, the end user is the last piece of the puzzle we need to pay attention to, as that’s what makes all this effort a task worth undertaking, even if most of them most likely don’t need what WebKit is, which is perfectly fine :-)

We like to make this distinction of 4 possible types of users explicit because we think it’s important to understand the complexity of the amount of use cases and the diversity of potential users and customers we need to provide service for, which is behind our decisions and the way we prioritize our work.

Strategic goals

Our main goal is that our product, the WebKit web engine, is useful for more and more people in different situations. Because of this, it is important that the platform is homogeneous and that it can be used reliably with all the engines available nowadays, and this is why compatibility and interoperability is a must, and why we work with the the standards bodies to help with the design and implementation of several Web specifications.

With WPE, it is very important to be able to run the engine in small embedded devices, and that requires good performance and being efficient in multiple hardware architectures, as well as great flexibility for specific hardware, which is why we provided WPE with a backend-based architecture, and reduced dependencies to a minimum.

Then, it is also important that the QA Infrastructure is good enough to keep the releases working and with good quality, which is why I regularly maintain, evolve and keep an eye on the EWS and post-commit bots that keep WebKitGTK and WPE building, running and passing the tens of thousands of tests that we need to check continuously, to ensure we don’t regress (or that we catch issues soon enough, when there’s a problem). Then of course it’s also important to keep doing security releases, making sure that we release stable versions with fixes to the different CVEs reported as soon as possible.

Finally, we also make sure that we keep evolving our tooling as much as possible (see for instance the release of the new SDK earlier this year), as well as improving the documentation for both ports.

Last, all this effort would not be possible if not because we also consider a goal of us to maintain an efficient collaboration with the rest of the WebKit community in different ways, from making sure we re-use and contribute to other ports as much code as possible, to making sure we communicate well in all the forums available (e.g. Slack, mailing list, annual meeting).

Contributions to WebKit in numbers

Well, first of all the usual disclaimer: number of commits is for sure not the best possible metric, and therefore should be taken with a grain of salt. However, the point here is not to focus too much on the actual numbers but on the more general conclusions that can be extracted from them, and from that point of view I believe it’s interesting to take a look at this data at least once a year.

Igalia contributions to WebKit (2024)

With that out of the way, it’s interesting to confirm that once again we are still the 2nd biggest contributor to WebKit after Apple, with ~13% of the commits landed in this past 12-month period. More specifically, we landed 2027 patches out of the 15617 ones that took place during the past year, only surpassed by Apple and their 12456 commits. The remaining 1134 patches were landed mostly by Sony, followed by RedHat and several other contributors.

Igalia contributions to WebKit (2024) Now, if we remove Apple from the picture, we can observe how this year our contributions represented ~64% of all the non-Apple commits, a figure that grew about ~11% compared to the past year. This confirms once again our commitment to WebKit, a project we started contributing about 14 years ago already, and where we have been systematically being the 2nd top contributor for a while now.

Main areas of work

The 10 main areas we have contributed to in WebKit in the past 12 months are the following ones:

Web platform
Graphics
Multimedia
JavaScriptCore
New WPE API
WebKit on Android
Quality assurance
Security
Tooling
Documentation

In the next sections I’ll talk a bit about what we’ve done and what we’re planning to do next for each of them.

Web Platform

content-visibility:auto

This feature allows skipping painting and rendering of off-screen sections, particularly useful to avoid the browser spending time rendering parts in large pages, as content outside of the view doesn’t get rendered until it gets visible.

We completed the implementation and it’s now enabled by default.

Navigation API

This is a new API to manage browser navigation actions and examine history, which we started working on in the past cycle. There’s been a lot of work happening here and, while it’s not finished yet, the current plan is that Apple will continue working on that in the next months.

hasUAVisualTransition

This is an attribute of the NavigateEvent interface, which is meant to be True if the User Agent has performed a visual transition before a navigation event. It was something that we have also finished implementing and is now also enabled by default.

Secure Curves in the Web Cryptography API

In this case, we worked on fixing several Web Interop related issues, as well as on increasing test coverage within the Web Platform Tests (WPT) test suites.

On top of that we also moved the X25519 feature to the “prepare to ship” stage.

Trusted Types

This work is related to reducing DOM-based XSS attacks. Here we finished the implementation and this is now pending to be enabled by default.

MathML

We continued working on the MathML specification by working on the support for padding, border and margin, as well as by increasing the WPT score by ~5%.

The plan for next year is to continue working on core features and improve the interaction with CSS.

Cross-root ARIA

Web components have accessibility-related issues with native Shadow DOM as you cannot reference elements with ARIA attributes across boundaries. We haven’t worked on this in this period, but the plan is to work in the next months on implementing the Reference Target proposal to solve those issues.

Canvas Formatted Text

Canvas has not a solution to add formatted and multi-line text, so we would like to also work on exploring and prototyping the Canvas Place Element proposal in WebKit, which allows better text in canvas and more extended features.

Graphics

Completed migration from Cairo to Skia for the Linux ports

If you have followed the latest developments, you probably already know that the Linux WebKit ports (i.e. WebKitGTK and WPE) have moved from Cairo to Skia for their 2D rendering library, which was a pretty big and important decision taken after a long time trying different approaches and experiments (including developing our own HW-accelerated 2D rendering library!), as well as running several tests and measuring results in different benchmarks.

The results in the end were pretty overwhelming and we decided to give Skia a go, and we are happy to say that, as of today, the migration has been completed: we covered all the use cases in Cairo, achieving feature parity, and we are now working on implementing new features and improvements built on top of Skia (e.g. GPU-based 2D rendering).

On top of that, Skia is now the default backend for WebKitGTK and WPE since 2.46.0, released on September 17th, so if you’re building a recent version of those ports you’ll be already using Skia as their 2D rendering backend. Note that Skia is using its GPU-based backend only on desktop environments, on embedded devices the situation is trickier and for now the default is the CPU-based Skia backend, but we are actively working to narrow the gap and to enable GPU-based rendering also on embedded.

Architecture changes with buffer sharing APIs (DMABuf)

We did a lot of work here, such as a big refactoring of the fencing system to control the access to the buffers, or the continued work towards integrating with Apple’s DisplayLink infrastructure.

On top of that, we also enabled more efficient composition using damaging information, so that we don’t need to pass that much information to the compositor, which would slow the CPU down.

Enablement of the GPUProcess

On this front, we enabled by default the compilation for WebGL rendering using the GPU process, and we are currently working in performance review and enabling it for other types of rendering.

New SVG engine (LBSE: Layer-Based SVG Engine)

If you are not familiar with this, here the idea is to make sure that we reuse the graphics pipeline used for HTML and CSS rendering, and use it also for SVG, instead of having its own pipeline. This means, among other things, that SVG layers will be supported as a 1st-class citizen in the engine, enabling HW-accelerated animations, as well as support for 3D transformations for individual SVG elements.

On this front, on this cycle we added support for the missing features in the LBSE, namely:

Implemented support for gradients & patterns (applicable to both fill and stroke)
Implemented support for clipping & masking (for all shapes/text)
Implemented support for markers
Helped review implementation of SVG filters (done by Apple)

Besides all this, we also improved the performance of the new layer-based engine by reducing repaints and re-layouts as much as possible (further optimizations still possible), narrowing the performance gap with the current engine for MotionMark. While we are still not at the same level of performance as the current SVG engine, we are confident that there are several key places where, with the right funding, we should be able to improve the performance to at least match the current engine, and therefore be able to push the new engine through the finish line.

General overhaul of the graphics pipeline, touching different areas (WIP):

On top of everything else commented above, we also worked on a general refactor and simplification of the graphics pipeline. For instance, we have been working on the removal of the Nicosia layer now that we are not planning to have multiple rendering implementations, among other things.

Multimedia

DMABuf-based sink for HW-accelerated video

We merged the DMABuf-based sink for HW-accelerated video in the GL-based GStreamer sink.

WebCodecs backend

We completed the implementation of audio/video encoding and decoding, and this is now enabled by default in 2.46. As for the next steps, we plan to keep working on the integration of WebCodecs with WebGL and WebAudio.

GStreamer-based WebRTC backends

We continued working on GstWebRTC, bringing it to a point where it can be used in production in some specific use cases, and we will still be working on this in the next months.

Other

Besides the points above, we also added an optional text-to-speech backend based on libspiel to the development branch, and worked on general maintenance around the support for Media Source Extensions (MSE) and Encrypted Media Extensions (EME), which are crucial for the use case of WPE running in set-top-boxes, and is a permanent task we will continue to work on in the next months.

JavaScriptCore

ARMv7/32-bit support:

A lot of work happened around 32-bit support in JavaScriptCore, especially around WebAssembly (WASM): we ported the WASM BBQJIT and ported/enabled concurrent JIT support, and we also completed 80% of the implementation for the OMG optimization level of WASM, which we plan to finish in the next months. If you are unfamiliar with what the OMG and BBQ optimization tiers in WASM are, I’d recommend you to take a look at this article in webkit.org: “Assembling WebAssembly“.

We also contributed to the JIT-less WASM, which is very useful for embedded systems that can’t support JIT for security or memory related constraints, and also did some work on the In-Place Interpreter (IPInt), which is a new version of the WASM Low-level interpreter (LLInt) that uses less memory and executes WASM bytecode directly without translating it to LLInt bytecode (and should therefore be faster to execute).

Last, we also contributed most of the implementation for the WASM GC, with the exception of some Kotlin tests.

As for the next few months, we plan to investigate and optimize heap/JIT memory usage in 32-bit, as well as to finish several other improvements on ARMv7 (e.g. IPInt).

New WPE API

The new WPE API is a new API that aims at making it easier to use WPE in embedded devices, by removing the hassle of having to handle several libraries in tandem (i.e. WPEWebKit, libWPE and WPEBackend-FDO, for instance), available from WPE’s releases page, and providing a more modern API in general, better aimed at the most common use cases of WPE.

A lot of effort happened this year along these lines, including the fact that we finally upstreamed and shipped its initial implementation with WPE 2.44, back in the first half of the year. Now, while we recommend users to give it a try and report feedback as much as possible, this new API is still not set in stone, with regular development still ongoing, so if you have the chance to try it out and share your experience, comments are welcome!

Besides shipping its initial implementation, we also added support for external platforms, so that other ones can be loaded beyond the Wayland, DRM and “headless” ones, which are the default platforms already included with WPE itself. This means for instance that a GTK4 platform, or another one for RDK could be easily used with WPE.

Then of course a lot of API additions were included in the new API in the latest months:

Screens management API: API to handle different screens, ask the display for the list of screens with their device scale factor, refresh rate, geometry…
Top level management API: This API allows a greater degree of control, for instance by allowing more than one WebView for the same top level, as well as allowing to retrieve properties such as size, scale or state (i.e. full screen, maximized…).
Maximized and minimized windows API: API to maximize/minimize a top level and monitor its state. mainly used by WebDriver.
Preferred DMA-BUF formats API: enables asking the platform (compositor or DRM) for the list of preferred formats and their intended use (scanout/rendering).
Input methods API: allows platforms to provide an implementation to handle input events (e.g. virtual keyboard, autocompletion, auto correction…).
Gestures API: API to handle gestures (e.g. tap, drag).
Buffer damaging: WebKit generates information about the areas of the buffer that actually changed and we pass that to DRM or the compositor to optimize painting.
Pointer lock API: allows the WebView to lock the pointer so that the movement of the pointing device (e.g. mouse) can be used for a different purpose (e.g. first-person shooters).

Last, we also added support for testing automation, and we can support WebDriver now in the new API.

With all this done so far, the plan now is to complete the new WPE API, with a focus on the Settings API and accessibility support, write API tests and documentation, and then also add an external platform to support GTK4. This is done on a best-effort basis, so there’s no specific release date.

WebKit on Android

This year was also a good year for WebKit on Android, also known as WPE Android, as this is a project that sits on top of WPE and its public API (instead of developing a fully-fledged WebKit port).

In case you’re not familiar with this, the idea here is to provide a WebKit-based alternative to the Chromium-based Web view on Android devices, in a way that leverages HW acceleration when possible and that it integrates natively (and nicely) with the several Android subsystems, and of course with Android’s native mainloop. Note that this is an experimental project for now, so don’t expect production-ready quality quite yet, but hopefully something that can be used to start experimenting with selected use cases.

If you’re adventurous enough, you can already try the APKs yourself from the releases page in GitHub at https://github.com/Igalia/wpe-android/releases.

Anyway, as for the changes that happened in the past 12 months, here is a summary:

Updated WPE Android to WPE 2.46 and NDK 27 LTS
Added support for WebDriver and included WPT test suites
Added support for instrumentation tests, and integrated with the GitHub CI
Added support for the remote Web inspector, very useful for debugging
Enabled the Skia backend, bringing HW-accelerated 2D rendering to WebKit on Android
Implemented prompt delegates, allowing implementing things such as alert dialogs
Implemented WPEView client interfaces, allowing responding to things such as HTTP errors
Packaged a WPE-based Android WebView in its own library and published in Maven Central. This is a massive improvement as now apps can use WPE Android by simply referencing the library from the gradle files, no need to build everything on their own.
Other changes: enabled HTTP/2 support (via the migration to libsoup3), added support for the device scale factor, improved the virtual on-screen keyboard, general bug fixing…

On top of that, we published 3 different blog posts covering different topics, from a general intro to a more deep dive explanation of the internals, and showing some demos. You can check them out in Jani’s blog at https://blogs.igalia.com/jani

As for the future, we’ll focus on stabilization and regular maintenance for now, and then we’d like to work towards achieving production-ready quality for specific cases if possible.

Quality Assurance

On the QA front, we had a busy year but in general we could highlight the following topics.

Fixed a lot of API tests failures in the bots that were limiting our test coverage.
Fixed lots of assertions-related crashes in the bots, which were slowing down the bots as well as causing other types of issues, such as bots exiting early due too many failures.
Enabled assertions in the release bots, which will help prevent crashes in the future, as well as with making our debug bots healthier.
Moved all the WebKitGTK and WPE bots to building now with Skia instead of Cairo. This means that all the bots running tests are now using Skia, and there’s only one bot still using Cairo to make sure that the compilation is not broken, but that bot does not run tests.
Moved all the WebKitGTK bots to use GTK4 by default. As with the move to Skia, all the WebKit bots running tests now use GTK4 and the only one remaining building with GTK3 does not run tests, it only makes sure we don’t break the GTK3 compilation for now.
Working on moving all the bots to use the new SDK. This is still work in progress and will likely be completed during 2025 as it’s needed to implement several changes in the infrastructure that will take some time.
General gardening and bot maintenance

In the next months, our main focus would be a revamp of the QA infrastructure to make sure that we can get all the bots (including the debug ones) to a healthier state, finish the migration of all the bots to the new SDK and, ideally, be able to bring back the ready-to-use WPE images that we used to have available in wpewebkit.org.

Security

The current release cadence has been working well, so we continue issuing major releases every 6 months (March, September), and then minor and unstable development releases happening on-demand when needed.

As usual, we kept aligning releases for WebKitGTK and WPE, with both of them happening at the same time (see https://webkitgtk.org/releases and https://wpewebkit.org/release), and then also publishing WebKit Security Advisories (WSA) when necessary, both for WebKitGTK and for WPE.

Last, we also shortened the time before including security fixes in stable releases this year, and we have removed support for libsoup2 from WPE, as that library is no longer maintained.

Tooling & Documentation

On tooling, the main piece of news is that this year we released the initial version of the new SDK, which is developed on top of OCI-based containers. This new SDK fixes the issues with the current existing approaches based on JHBuild and flatpak, where one of them was great for development but poor for testing and QA, and the other one was great for testing and QA, but not very convenient for development.

This new SDK is regularly maintained and currently runs on Ubuntu 24.04 LTS with GCC 14 & Clang 18. It has been made public on GitHub and announced to the public in May 2024 in Patrick’s blog, and is now the officially recommended way of building WebKitGTK and WPE.

As for documentation, we didn’t do as much as we would have liked here, but we still landed a few contributions in docs.webkit.org, mostly related to WebKitGTK (e.g. Releases and Versioning, Security Updates, Multimedia). We plan to do more on this regard in the next months, though, mostly by writing/publishing more documentation and perhaps also some tutorials.

Final thoughts

This has been a fairly long blog post but, as you can see, it’s been quite a year for WebKit here at Igalia, with many exciting changes happening at several fronts, and so there was quite a lot of stuff to comment on here. This said, you can always check the slides of the presentation in the WebKit Contributors Meeting here if you prefer a more concise version of the same content.

In any case, what’s clear it’s that the next months are probably going to be quite interesting as well with all the work that’s already going on in WebKit and its Linux ports, so it’s possible that in 12 months from now I might be writing an equally long essay. We’ll see.

Thanks for reading!

Cross-compiling WebKit2GTK+ for ARM

Published September 30, 2016 / by mario / 2 Comments on Cross-compiling WebKit2GTK+ for ARM

I haven’t blogged in a while -mostly due to lack of time, as usual- but I thought I’d write something today to let the world know about one of the things I’ve worked on a bit during this week, while remotely attending the Web Engines Hackfest from home:

Setting up an environment for cross-compiling WebKit2GTK+ for ARM

I know this is not new, nor ground-breaking news, but the truth is that I could not find any up-to-date documentation on the topic in a any public forum (the only one I found was this pretty old post from the time WebKitGTK+ used autotools), so I thought I would devote some time to it now, so that I could save more in the future.

Of course, I know for a fact that many people use local recipes to cross-compile WebKit2GTK+ for ARM (or simply build in the target machine, which usually takes a looong time), but those are usually ad-hoc things and hard to reproduce environments locally (or at least hard for me) and, even worse, often bound to downstream projects, so I thought it would be nice to try to have something tested with upstream WebKit2GTK+ and publish it on trac.webkit.org,

So I spent some time working on this with the idea of producing some step-by-step instructions including how to create a reproducible environment from scratch and, after some inefficient flirting with a VM-based approach (which turned out to be insanely slow), I finally settled on creating a chroot + provisioning it with a simple bootstrap script + using a simple CMake Toolchain file, and that worked quite well for me.

In my fast desktop machine I can now get a full build of WebKit2GTK+ 2.14 (or trunk) in less than 1 hour, which is pretty much a productivity bump if you compare it to the approximately 18h that takes if I build it natively in the target ARM device I have :-)

Of course, I’ve referenced this documentation in trac.webkit.org, but if you want to skip that and go directly to it, I’m hosting it in a git repository here: github.com/mariospr/webkit2gtk-ARM.

Note that I’m not a CMake expert (nor even close) so the toolchain file is far from perfect, but it definitely does the job with both the 2.12.x and 2.14.x releases as well as with the trunk, so hopefully it will be useful as well for someone else out there.

Last, I want to thanks the organizers of this event for making it possible once again (and congrats to Igalia, which just turned 15 years old!) as well as to my employer for supporting me attending the hackfest, even if I could not make it in person this time.

Attending the Web Engines Hackfest

Published November 26, 2015 / by mario / 1 Comment on Attending the Web Engines Hackfest

It’s certainly been a while since I attended this event for the last time, 2 years ago, when it was a WebKitGTK+ only oriented hackfest, so I guess it was a matter of time it happened again…

It will be different for me this time, though, as now my main focus won’t be on accessibility (yet I’m happy to help with that, too), but on fixing a few issues related to the WebKit2GTK+ API layer that I found while working on our platform (Endless OS), mostly related to its implementation of accelerated compositing.

Besides that, I’m particularly curious about seeing how the hackfest looks like now that it has broaden its scope to include other web engines, and I’m also quite happy to know that I’ll be visiting my home town and meeting my old colleagues and friends from Igalia for a few days, once again.

Last, I’d like to thank my employer for sponsoring this trip, as well as Igalia for organizing this event, one more time.

See you in Coruña!

Importing include paths in Eclipse

Published November 7, 2015 / by mario / 1 Comment on Importing include paths in Eclipse

First of all, let me be clear: no, I’m not trying to leave Emacs again, already got over that stage. Emacs is and will be my main editor for the foreseeable future, as it’s clear to me that there’s no other editor I feel more comfortable with, which is why I spent some time cleaning up my .emacs.d and making it more “manageable”.

But as much as like Emacs as my main “weapon”, I sometimes appreciate the advantages of using a different kind of beast for specific purposes. And, believe me or not, in the past 2 years I learned to love Eclipse/CDT as the best work-mate I know when I need some extra help to get deep inside of the two monster C++ projects that WebKit and Chromium are. And yes, I know Eclipse is resource hungry, slow, bloated… and whatnot; but I’m lucky enough to have fast SSDs and lots of RAM in my laptop & desktop machines, so that’s not really a big concern anymore for me (even though I reckon that indexing chromium in the laptop takes “quite some time”), so let’s move on :-)

However, there’s this one little thing that still bothers quite me a lot of Eclipse: you need to manually setup the include paths for the external dependencies not in a standard location that a C/C++ project uses, so that you can get certain features properly working such as code auto-completion, automatic error-checking features, call hierarchies… and so forth.

And yes, I know there is an Eclipse plugin adding support for pkg-config which should do the job quite well. But for some reason I can’t get it to work with Eclipse Mars, even though others apparently can use it there for some reason (and I remember using it with Eclipse Juno, so it’s definitely not a myth).

Anyway, I did not feel like fighting with that (broken?) plugin, and in the other hand I was actually quite inclined to play a bit with Python so… my quick and dirty solution to get over this problem was to write a small script that takes a list of package names (as you would pass them to pkg-config) and generates the XML content that you can use to import in Eclipse. And surprisingly, that worked quite well for me, so I’m sharing it here in case someone else finds it useful.

Using frogr as an example, I generate the XML file for Eclipse doing this:

  $ pkg-config-to-eclipse glib-2.0 libsoup-2.4 libexif libxml-2.0 \
        json-glib-1.0 gtk+-3.0 gstreamer-1.0 > frogr-eclipse.xml

…and then I simply import frogr-eclipse.xml from the project’s properties, inside the C/C++ General > Paths and Symbols section.

After doing that I get rid of all the brokenness caused by so many missing symbols and header files, I get code auto-completion nicely working back again and all those perks you would expect from this little big IDE. And all that without having to go through the pain of defining all of them one by one from the settings dialog, thank goodness!

Now you can quickly see how it works in the video below:

VIDEO: Setting up a C/C++ project in Eclipse with pkg-config-to-eclipse

This has been very helpful for me, hope it will be helpful to someone else too!

On Linux32 chrooted environments

Published July 3, 2015 / by mario / 2 Comments on On Linux32 chrooted environments

I have a chrooted environment in my 64bit Fedora 22 machine that I use every now and then to work on a debian-like 32bit system where I might want to do all sorts of things, such as building software for the target system or creating debian packages. More specifically, today I was trying to build WebKitGTK+ 2.8.3 in there and something weird was happening:

The following CMake snippet was not properly recognizing my 32bit chroot:

string(TOLOWER ${CMAKE_HOST_SYSTEM_PROCESSOR} LOWERCASE_CMAKE_HOST_SYSTEM_PROCESSOR)
if (CMAKE_COMPILER_IS_GNUCXX AND "${LOWERCASE_CMAKE_HOST_SYSTEM_PROCESSOR}" MATCHES "(i[3-6]86|x86)$")
    ADD_TARGET_PROPERTIES(WebCore COMPILE_FLAGS "-fno-tree-sra")
endif ()

After some investigation, I found out that CMAKE_HOST_SYSTEM_PROCESSOR relies on the output of uname to determine the type of the CPU, and this what I was getting if I ran it myself:

(debian32-chroot)mario:~ $ uname -a
Linux moucho 4.0.6-300.fc22.x86_64 #1 SMP Tue Jun 23 13:58:53 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux

Let’s avoid nasty comments about the stupid name of my machine (I’m sure everyone else uses clever names instead), and see what was there: x86_64.

That looked wrong to me, so I googled a bit to see what others did about this and, besides finding all sorts of crazy hacks around, I found that in my case the solution was pretty simple just because I am using schroot, a great tool that makes life easier when working with chrooted environments.

Because of that, all I would have to do would be to specify personality=linux32 in the configuration file for my chrooted environment and that’s it. Just by doing that and re-entering in the “jail”, the output would be much saner now:

(debian32-chroot)mario:~ $ uname -a
Linux moucho 4.0.6-300.fc22.x86_64 #1 SMP Tue Jun 23 13:58:53 UTC 2015
i686 i686 i686 GNU/Linux

And of course, WebKitGTK+ would now recognize and use the right CPU type in the snippet above and I could “relax” again while seeing WebKit building again.

Now, for extra reference, this is the content of my schroot configuration file:

$ cat /etc/schroot/chroot.d/00debian32-chroot
[debian32-chroot]
description=Debian-like chroot (32 bit) 
type=directory
directory=/schroot/debian32/
users=mario
groups=mario
root-users=mario
personality=linux32

That is all, hope somebody else will find this useful. It certainly saved my day!

WebKitGTK+ Hackfest 2013: The Return of the Thing

Published December 13, 2013 / by mario

As many other WebKitGTK+ hackers (30 in total), I flew last Saturday to A Coruña to attend the 5th edition of the WebKitGTK+ Hackfest, hosted once again by Igalia at their premises and where people from several different affiliations gathered together to try to give our beloved port a boost.

As for me, I flew there to work mainly on accessibility related issues, making the most of the fact that both Joanie (Orca maintainer) and Piñeiro (ATK maintainer) would be there too, so it should be possible to make things happen faster, specially discussion-wise.

And turns out that, even if I feel like I could have achieved more than what I actually did (as usual), I believe we did quite well in the end: we discussed and clarified things that were blocking the mapping of new WAI-ARIA roles in WebKitGTK+, we got rid of a bunch of WebKit1-specific unit tests (Joanie converted them into nice layout tests that will be run by WebKit2GTK+ too), we got a few new roles in ATK to be able to better map things from the web world and and we fixed a couple of issues in the way too.

Of course, not everything has been rainbows and unicorns, as it seems that one of the patches I landed broke the inspector for WebKit2GTK+ (sorry Gustavo!). Fortunately, that one has been rolled out already and I hope I will be able to get back to it soon (next week?) to provide a better patch for that without causing any problem. Fingers crossed.

In the other hand, my mate Brian Holt joined us for three days too and, despite of being his first time in the hackfest, he got integrated pretty quickly with other hackers, teaming up to collaborate in the big boost that the network process & multiple web processes items have went through during the event. And not only that, he also managed to give a boost to his last patch to provide automatic memory leak detection in WebKitGTK+, which I’m sure it will be a great tool once it’s finished and integrated upstream.

Anyway, if you want more details on those topics, or anything else, please check out the blog posts that other hackers have been posting these days, specially Carlos’s blog post, which is quite extensive and detailed.

Of course, I would like to thank the main sponsors Igalia and the GNOME Foundation for making this thing happen again, and to my employer Samsung for helping as well by paying our trips and accommodation, as well as the snacks and the coffee that helped us stay alive and get fatter during the hackfest.

Last, I would like to mention (in case anyone reading this wondered) that it has indeed felt a bit strange to go the city where I used to live in and stay in a hotel, not to mention going to the office where I used to work in and hang around it as a visitor. However, both my former city and my former colleagues somehow ensured that I felt as “at home” once again, and so I can’t do anything about it but feeling enormously grateful for that.

Thank you all, and see you next year!

Greppin’ in the past with git

Published November 14, 2013 / by mario / 4 Comments on Greppin’ in the past with git

It seems that one can never stop learning new things with git, no matter for how long you’ve been using it (in my case, I’m a proud git user since 2008), because today I added a new trick to my toolbox, that already proved to be quite useful: “grepping” files in a git repository, as you would do it with git grep, but using a commit-id to limit the search to a specific snapshot of your project.

In other words, I found that it’s possible to do things like, say, grep files to search for something in your repository considering how it was, say, some commits ago.

This is the “magical” command:

git grep <search-params> <tree-id>

This is what I get if I try to search for updateBackingStore() in my local clone of WebKit, as if my current branch was “50 commits older” than what it actually is:

$ git grep updateBackingStore HEAD~50
0ae236137d560da6ca889a826a8f3d023364a669:AccessibilityObject.cpp:void AccessibilityObject::updateBackingStore()
0ae236137d560da6ca889a826a8f3d023364a669:AccessibilityObject.h:    void updateBackingStore();
0ae236137d560da6ca889a826a8f3d023364a669:AccessibilityObject.h:inline void AccessibilityObject::updateBackingStore() { }
0ae236137d560da6ca889a826a8f3d023364a669:atk/WebKitAccessibleUtil.h:        coreObject->updateBackingStore(); \
0ae236137d560da6ca889a826a8f3d023364a669:atk/WebKitAccessibleUtil.h:        coreObject->updateBackingStore(); \
0ae236137d560da6ca889a826a8f3d023364a669:atk/WebKitAccessibleWrapperAtk.cpp:    coreObject->updateBackingStore();
0ae236137d560da6ca889a826a8f3d023364a669:ios/WebAccessibilityObjectWrapperIOS.mm:    m_object->updateBackingStore();
0ae236137d560da6ca889a826a8f3d023364a669:ios/WebAccessibilityObjectWrapperIOS.mm:    m_object->updateBackingStore();
0ae236137d560da6ca889a826a8f3d023364a669:mac/WebAccessibilityObjectWrapperBase.mm:    // Calling updateBackingStore() can invalidate this element so self must be retained.
0ae236137d560da6ca889a826a8f3d023364a669:mac/WebAccessibilityObjectWrapperBase.mm:    m_object->updateBackingStore();

I don’t know about you, but I find this quite useful for me to answer questions such as “Where was this function being used in commit X?”, and things like that.

Anyway, you might have noticed that I mentioned <tree-id> in the recipe instead of <commit-id>, yet I used HEAD~50 in the example, which is actually a commit-id. And still works.

And the short explanation, without trying to explain here all the different kind of data types that git keeps internally for every repository (mainly commits, trees and blobs), is that git is smart enough to find the right tree-id associated to a given commit-id by just considering the current path inside the repository and the tree-id associated to the top directory for a given commit.

But how to know that tree-id myself in case I want to? Easy, just pretty print the full information of the commit object you’re interested in, instead of only seeing the abbreviated version (what you usually see with git show or git log:

$ git cat-file -p HEAD~50
tree 0ae236137d560da6ca889a826a8f3d023364a669
parent bdb7a7949a29988da3fe50a65d6c694d5084d379
author [...]

See that tree thing in the first line? That’s the tree-id that git needs for grepping, which as you can see can be easily extracted from a particular commit. Actually, you could get easily the tree-id for any subdirectory from this point, by using the git ls-tree command:

$ git ls-tree 0ae236137d560da6ca889a826a8f3d023364a669
100644 blob 3fe2340c9614e893f0dfeb720f23773bbf1ea076	.dir-locals.el
100644 blob 741c4d53b5a0338cf36900a283e89408d0f9d457	.gitattributes
100644 blob f45a975762be9a429aa971c18da01b433c559553	.gitignore
100644 blob d571aa28ea86c14c7880533bf3ba68e9ef4b3c81	.qmake.conf
100644 blob 10f85055ae9f3823f0d20808599f644c18af7921	CMakeLists.txt
100644 blob 5eb66e7bcbc7543eb3a4dbf183a9043545776659	ChangeLog
100644 blob 7dbe9d2e0029bab47b8b2b22065a1032ecfe4434	ChangeLog-2012-05-22
040000 tree d42a0b3121ed7993cfd250426d20472769760f87	Examples
100644 blob 78d89e5c70ad56c38b0c25e7705d42fa380c4ee0	GNUmakefile.am
040000 tree 4a9e87fc1f35efa1349a18b1df694530c981c57e	LayoutTests
100644 blob 14e33157011157797dac62c494bac0bf254d7c2f	Makefile
100644 blob ee723d830dea51d1ce9e2d1ad8c985eeca2d4f3f	Makefile.shared
040000 tree 20c763d6a4e8749ad9e041e8372e9f47dc722f45	ManualTests
040000 tree 660d88b926cf618ab9e1612b8e2a3e97b15dbcbe	PerformanceTests
040000 tree fbf9703d3e9a9e4cf2ff10817c99ba3a5de87410	Source
040000 tree 346110c441a674334f5f56ef42b9dd40def89c76	Tools
040000 tree 262cb11d9b491be35daee570f9b825bce5715579	WebKit.xcworkspace
040000 tree b9e48a7a24b4973b253ee14053808b40d67c94aa	WebKitLibraries
040000 tree adce37b690957abdd21d2dd8ff77302c5a5a9071	Websites
100755 blob befd429487fc5ac9bb3494800f4eeaef1e607663	autogen.sh

And of course, “navigating” with more calls to git ls-tree you could also get the tree-id for a specific subdirectory, in case you wanted to constraint the search to that specific path of your repo.

However, considering that git is so good at translating a commit-id into a tree-id, my personal recommendation is that, instead, you first cd into the path you want to focus the search in, and then let git do its “magic” by just using the git grep <search-params> <tree-id> command.

So that’s it. Hope you find this useful, and please do not hesitate to share any comment or suggestion you might have with regard to this or any other “git trick” you might know.

I honestly love using git so much that sometimes I wonder if coding is not just a poor excuse to use git. Probably not, but the thing is that I can not imagine my life without it anymore. That’s a fact.

Goodbye Pango! Goodbye GAIL!

Published September 13, 2013 / by mario / 2 Comments on Goodbye Pango! Goodbye GAIL!

As I mentioned in my previous post before GUADEC, I’ve been putting some effort lately on trying to improve the accessibility layer of WebKitGTK+, as part of my work here at Samsung, and one of the main things I’ve worked on was the removal of the dependency we had on Pango and GAIL to implement the atk_text_get_text_*_offset() family of functions for the different text boundaries.

And finally, I’m really happy to say that such a task is complete once and for all, meaning that now those functions should work as well or as bad on WebKit2GTK+ as they do in WebKitGTK+, so the weird behaviour described in bug 73433 is no longer an issue. You can check I’m not lying by just taking a look to the commit that removed both all trace of Pango and GAIL in the code, as well as and the one that removed the GAIL dependency from the build system. And if you want more detail, just feel free to check the whole dependency tree in WebKit’s bugzilla.

This task has been an interesting challenge for me indeed, and not only because it was one of the biggest accessibility related tasks I’ve worked on in WebKitGTK+ since late 2012 (so I needed to get my brain trained again on it), but also because reimplementing these functions forced me to dive into text editing and accessibility code in WebCore as I never did before. And it’s so cool to see how, despite of having to deal eventually with the frustrating feeling of hitting my head against a wall, at the end of the day it all resulted on a nice set of patches that do the work and help advance the state of the ATK based accessibility layer in WebKitGTK+ forward.

Anyway, even though that is probably the thing that motivated me to write this blog post, that was not the only thing that I did since GUADEC (which has been a blast, by the way):

On the personal side, I’ve spent two lovely weeks in Spain on holidays, which was the biggest period of time I’ve been outside of the UK since I arrived here, and had an amazing time there just “doing nothing”(tm) but lying around on the beach and seeing the grass grow. And there is not much grass there anywhere, so you can imagine how stressful that life was… it was great.

In the other hand, on the professional side, I’d say that one of the other big things that happened to me lately was that I finally became accepted as a WebKit reviewer, meaning that now I can not only help breaking the Web, but also authorize others to do it so. And while agree that might be fun in some way, it probably would not be very cool, so forgive me if I try instead to do my best to help get exactly the opposite result: make things work better.

And truth to be told, this “upgrade” came just with perfect timing, since these days quite some work is being done in the accessibility layer for both the WebKitGTK+ and the WebKitEFL ports thanks also to my mates Denis Nomiyama, Anton Obzhirov, Brian Holt and Krzysztof Czech from Samsung, and that work would ideally need to be reviewed by someone familiar with the ATK/AT-SPI based accessibility stack. And while I’m still by far not the most knowledgeable person in the world when it comes to those topics, I believe I might have a fairly well knowledge about it anyway, so I assume (and hope) that my reviews will certainly add value and help with those specific pieces of work.

And as a nice plus, now I can finally “return the favour” to the only accessibility reviewer WebKit had until now (Chris Fleizach) by helping reviewing his patches as well, in a similar fashion to what he has been tirelessly doing for me for the last 3 years and a half. Yay!

To finish , I’d like to get back again to the original topic of this post and say a big “thank you” to everyone who helped me along the way with the removal of Pango and GAIL from the ATK specific code. Special thanks go to those who spend time performing the code reviews, as it’s the case of Martin, Gustavo and Chris. I wouldn’t be writing this post otherwise.

Thanks!

I’m going to GUADEC!

Published July 12, 2013 / by mario / 1 Comment on I’m going to GUADEC!

One year again GUADEC is approaching and, also again, I’m very happy to say that I’ll be there as well this time, even if I have to recognize it was not on my plans for this year, at least not initially.

And the reason why it was not initially in my plans was mainly because I’ve been already through quite some changes during these past months year, and my family just came over to the UK two months ago. This means that, even I already arrived by the beginning of the year, we just started to settle here as a family a few weeks ago. So in that context, I didn’t feel like leaving them alone for one week already now, it definitely would look like a “wrong management of priorities” to me.

However, it turns out that my wife and kids won’t be here anyway during the first week of August and, on top of that, Samsung has been so kind to sponsor this trip just based on the simple fact that I’m part of the GNOME community. So, I certainly can no longer find a single reason not to go and spend 7 amazing days in Brno, meeting people that I normally see only in conferences (and this time that group of people will be bigger than ever, since my former mates from Igalia are now also included there), while attending to what it seems to be a very appealing event.

Also, I will try to make the most of the trip to do some work during the different hackfests and BoFs that are already planned, which special focus in the one about accessibility, of course. As a personal goal, I expect to have the chance to move forward some work I’ve been doing lately in the WebKitGTK+ a11y world, such as getting rid of the nasty dependency on Pango/Gail we still have there, something I’ve been already working on for some time now, and which I expect it will be fixed soon, hopefully before GUADEC, although time will tell.

Once that it’s fixed, WebKit2GTK+ based apps should recover the ability to properly expose text through the atk_text_get_text_*_offset() family of functions for different text boundaries, which means that ATs (e.g. the Orca screen reader) will be able to properly allow again line-by-line navigation when in caret browsing mode. And, as you can imagine, this is quite a big problem these days, since WebKit2GTK+ that has become the default backend for some core apps such as the Epiphany browser with the GNOME 3.8 release, so fixing this is like a high priority now, I’d say.

Anyway, I’m starting to write too much (as usual) for what it was going to be a short “I’m going to GUADEC” blog post, so I will stop right now, although not without first thanking Samsung for sponsoring this my first trip to the Czech Republic.

See you all in three weeks!

WebKit Contributors Meeting 2013

Published May 3, 2013 / by mario / 2 Comments on WebKit Contributors Meeting 2013

It turns out I’m writing this post at 6:00 AM in the morning from a hotel instead of doing it at a more reasonable time from my comfy home or a nice cafeteria. That’s already quite a new thing by itself, and the reason for that is not that I became crazy or something, but the fact that I’m completely jet-lagged in California right now in order to attend my second WebKit Contributors Meeting (my first time was in 2011), this time as part of the Samsung team in the UK R&D center, together with my mate Anton Obzhirov.

With regard to that, it has been a very interesting experience so far where I could meet new people I still haven’t had the chance to see in real life yet (e.g. my mates from other Samsung R&D centers or some guys from Apple I didn’t have the chance to meet in person before), as well as chat again with some friends and former mates that I haven’t seen for a while, such as Martin, Xan and Philippe from Igalia, Byungseon from LG, Nayan from Motorola or Gustavo from Collabora to mention some of them. It’s strange, and at the same time wonderful, how easily you can catch up on conversations with people that you barely see once a year (or even less) and mainly in conferences, and definitely one of my favourite parts of attending these kind of events, to be honest.

Also, from a less social point of view, I have to say I found very interesting the sessions I’ve attended so far, specially the one about “managing the differences between ports”, although the one about “build systems” was quite interesting too. Not sure how far we are yet in the WebKitGTK+ port from realistically switching to some kind of commonly agreed build system (cmake?), but at least it’s a good start to agree on the fact that it would be an interesting move and now that some people pushing for it.

My only regret about this first day is that I missed Hyatt‘s talk about pagination due to some health issues I’m experimenting while in California, mostly due to the extremely hot and dry weather (anything over 25 Celsius is “unbearable hot” for me), which is causing me a little bit of cough, sore throat and fever, all well mixed with the jet lag to make it a perfect “welcome pack” to the meeting. Fortunately, I got some “interesting” medicines that seem to have relieved a bit the pain and I could attend the rest of the sessions without much trouble, other than some occasional coughing. Not bad.

By the way, for those of you who were not lucky enough to attend the meeting but are anyway interested in the topics being discussed here, make sure you check the main TRAC page for the meeting, where you can also find transcripts for most of the sessions.

As for today, some more sessions will take place as well as a couple of hackathons so I expect it to be very interesting as well. Also I hope I can find some time too to work a bit on my patches to remove the nasty dependency on pango we have in WebKitGTK+ accessibility code, which is preventing us to have proper caret navigation in WebKit2GTK+ based browsers, as well as to discuss possible ways in which our lab could collaborate more actively upstream. Seems a promising day already!

Last (but not least), and in a completely unrelated and super-off-topic way, I would like to tell the world that I’m extremely happy for the fact that next week will be the end of my “lonely existence in the UK”, finally. After 4 months of living away from my family with just some flash trips from Friday to Sunday (every 2 weeks), I’m once and for all travelling on Thursday to my home town with a one way plane ticket to do some final arrangements, put everything (family included!) in the car and travel to Santander, where we’ll be taking a ferry that will take us to the Portsmouth (southern coast of England), from where we will just drive to the UK in order to start our new life, all together again.

It has been quite hard for us to live this way for so long, but I think in the end we managed to handle the situation quite well, and now it seems all our efforts are already paying off because things seem to be finally fitting in the right places: we have a lovely house, we have a place in a nearby public school for my oldest kid to start on September, most of the needed paperwork seems to be done and we already moved all our stuff from Spain (lots of toys!), which is now waiting to be used in our new place.

I really can’t wait to live again in the noisy and chaotic atmosphere that two kids can so easily create around them. Even if that means it will probably drive me crazy every now and then and that I won’t sleep that well sometimes.

Yes. Even considering that.

Accessibility in [WebKit]GTK+

Published February 3, 2013 / by mario / 8 Comments on Accessibility in [WebKit]GTK+

This past week I’ve spent some time explaining to my mates at Samsung the basics about how accessibility works and is implemented in WebKitGTK+. I realized, yet again, of how messy and confusing everything can be the first time you encounter these things. After all, WebKit is quite a complex project already and accessibility is not a simple matter either.

In order to help them better understand this topic, I wrote a summary to have as reference that explains in my own words which the main pieces of the whole puzzle are, and how they relate to one another. In my experience, it’s not always easy to understand the big picture quickly, and I think this kind of documentation can be quite useful for anyone willing to contribute to accessibility in WebKitGTK+. At least it would have been useful for me when I started working on this. I only regret not having written it before, but better late than never, right?

So let’s begin then. I will start by talking about accessibility-only stuff, which are basically common to any accessible application based in GTK+. Then I will explain the bits specific to WebKitGTK+ and how they fit in the picture.

Accessibility in GTK+ applications

The parts, or “actors”, involved in any GTK+ application from an accessibility point of view are:

Assistive Technologies (ATs)
AT-SPI (Assistive Technology Service Provider Interface)
ATK (Accessibility ToolKit)
ATK <-> AT-SPI bridge
GTK+
GTK’s Accessibility Implementation

Now let’s describe all those points, one by one:

Assistive Technologies (ATs):

ATs are applications whose main purpose is to facilitate access and/or interaction with certain bits of information interesting from an accessibility related point of view, exposed by other applications. This access/interaction can be primarily output based. For instance the Orca screen reader is an AT which provides access via text to speech and/or refreshable braille to on-screen information exposed by editors, browsers, mail agents and other applications.

Other ATs are primarily input based, allowing the user to interact with the exposed applications by executing certain actions over them (e.g. clicking on a exposed link), so it’s not just about “consuming” information. Normally, ATs are called the clients and the applications exposing information the servers, as in the end it’s actually an implementation of a typical client/server architecture.

AT-SPI (Assistive Technology Service Provider Interface):

Set of interfaces that Assistive Technologies (the clients) understand and use to inspect and interact with the accessible content exposed by applications in Linux environments. At some point, “someone” has to provide actual AT-SPI objects (linked together forming a AT-SPI hierarchy) implementing several of those interfaces (depending of the type of object) so ATs can “see them”.

This is the job of the AT-SPI registry, a daemon which takes care of maintaining a hierarchy of AT-SPI objects for every single accessible application in the system, in a centralized way, so ATs can interact with them. It is worth mentioning that the parent/children relationships in that hierarchy are modelled in terms of D-Bus, so different AT-SPI objects can belong to different processes.

ATK (Accessibility ToolKit):

The toolkit used by GTK+ applications to expose accessible representations of the toolkit’s objects, along with appropriate interfaces, on the side of the applications exposing content (the servers). This representation is an almost a 1:1 match with the objects and interfaces defined by AT-SPI (that is, almost).

The main difference when it comes to understanding its place in the puzzle is that AT-SPI is what clients (ATs) understand, and that is not process-bounded (see previous point). ATK, in contrast, is what servers implement to expose accessible information, and it is process-bounded. Thus the parent/children relationships in the ATK hierarchy are modelled by actual references (pointers) between objects living in the same process.

ATK <-> AT-SPI bridge:

The glue that makes sure there’s a mapping between the ATK hierarchy living in the server process and the AT-SPI hierarchy held by the AT-SPI registry. Such a bridge is implemented in terms of D-Bus too, as it needs to communicate with the registry whenever something needs to be updated there, as well as when the server needs to react to external actions coming from ATs (e.g. perform the default action for an object).

GTK+:

The widgets toolkit normally used by applications embedding WebKitGTK+. Explaining what GTK+ is beyond the scope of this post, so I will assume you already know what it is.

GTK’s Accessibility Implementation:

Provides ATK objects implementing different ATK interfaces for every widget from the GTK+ library, and uses the ATK <-> AT-SPI bridge to communicate with the AT-SPI registry. This means that if you use standard GTK+ widgets only, your application will be accessible out-of-the-box. On the contrary, should you use custom widgets, you’ll probably have to write custom ATK objects implementing the proper ATK interfaces to make them accessible too.

So that’s all so far, when it comes to GTK+ applications. Check the following diagram for a more detailed look at all these hierarchies for a hypothetical GTK+ application exposing information and a screen reader accessing it:

As you can see, there’s an ATK tree matching the GTK+ hierarchy, and another AT-SPI tree matching the ATK one. Finally, the screen reader accesses the information through that AT-SPI tree, as explained above.

Accessibility in WebKitGTK+

Now that we already understand the basics of accessibility in GTK+ applications, let’s add the bits specifically related to WebKitGTK+. Namely:

WebCore’s Accessibility Objects
WebKitGTK+ (ATK) wrappers
WebKit2GTK+ specific details

Again, a picture is usually better than just text, so here you have one too:

In order to clarify it a bit more before explaining each point, let’s just say that you’ll have to look in the dashed box named WebCore accessibility world, where the hierarchy on the left (red & orange) represent the WebCore Accessibility objects, while the one on the right (the green one) represents the WebKitGTK+ ATK wrappers.

With this in mind, let’s examine these three points in more depth:

WebCore Accessibility objects:

Similar to GTK’s Accessibility Implementation, WebCore‘s accessibility objects are the implementation of an independent hierarchy exposing accessibility related information for objects present in a web page. As the mission of accessibility in WebKit is to expose information to users that are normally being rendered in the screen (as well as some other information that might be hidden to regular users), there is a tight relationship between this hierarchy and other ones in WebKit, such as the DOM tree and the Render Objects tree.

This layer is meant to be platform-agnostic, so you won’t find much WebKitGTK+ specific stuff here. Instead, you will find the implementation of the accessibility related specifications published by the W3C‘s Web Accessibility Initiative (WAI), such as WAI-ARIA.

WebKitGTK+ ATK wrappers:

An ATK-based implementation of an accessibility hierarchy where every ATK object will take care of wrapping the proper accessibility object from WebCore, as well as implementing the proper ATK interfaces depending on the situation (e.g. the role of the WebCore accessibility object, some properties coming from the associated Render Object…).

The ATK hierarchy created here is connected with the ATK hierarchy from the embedding application (normally a GTK+ app) by setting the root ATK object in this tree (normally representing DOM‘s root element) as the child of the leaf ATK object in the tree coming from the embedding application (normally the GtkWidget containing the WebView).

As is the case with any other regular GTK+ application, this ATK hierarchy will finally be seen by ATs thanks to the translation that the ATK <-> AT-SPI bridge will do for us, making the whole ATK tree from the WebKitGTK+ based application (from the top level GTK+ window down to the deepest accessibility object inside WebCore) available to the AT-SPI registry by means of D-Bus.

WebKit2GTK+ specific details:

I already talked about this in previous posts, so I will focus here just on commenting the main difference compared to the generic case for WebKitGTK+ described earlier (see previous diagram above):

WebKit2GTK+ implements a split-process model, where the high level API belongs to one process (the UI process) while the core logic of the web engine lives in another one (the Web process).

From an accessibility point of view, this means that the full hierarchy of ATK objects we had before is also split in two parts: some accessibility objects are now in the UI process and the rest of them will be in the Web process.

To be more specific, we’ll find the following objects in each process:

UI Process: basically, the ATK object associated with the GtkWidget provided by WebKitGTK+, that is, the WebView.
Web Process: the whole hierarchy of ATK objects wrapping the accessibility objects from WebCore.

As I explained previously, these two ATK hierarchies will be seen as a single accessibility hierarchy by ATs thanks to the “magic” of AtkPlug and AtkSocket classes, which takes care of exposing everything together in a single AT-SPI tree. And remember that such a tree is modelled by means of D-Bus, so it does not matter that things are actually in different processes.

Thus, since ATs just understand AT-SPI, they will see The Right Thing ™ as in the previous case where we have one single process. See the following diagram for a more visual explanation of this:

Wrapping up

So that’s it. At the end the post turned out to be longer than what I was expecting, as my initial idea was to publish the stuff I wrote internally at Samsung this week, but ended up extending it quite a lot!

At least I hope this will be helpful for anyone willing to contribute to accessibility, either in WebKitGTK+ or in a more general way.

After all, most of the stuff I talked about here applies to every accesible GTK+ application: Assistive Technologies (ATs), AT-SPI, ATK, the ATK <-> AT-SPI bridge…

Last, I would like to thanks Joanmarie Diggs from Igalia for her help with this blog post. One certainly feels more confident writing a long article like this one about a very specific topic when you have one of the most experienced persons on the matter reviewing it!

My first week at SERI

Published January 12, 2013 / by mario / 9 Comments on My first week at SERI

So, after almost 3 months of “holidays”, I’ve finally started working on my new job this Tuesday in Samsung Electronics Research Institute UK (aka SERI), where I’ll join a team mainly working in A/V and DTV related stuff while, at the same time, I’ll keep contributing to WebKit and WebKitGTK+.

As you can imagine, being the first week means that I mostly spent my time learning a lot of stuff about my new job and the tools I’d be using, as well as setting up my development environment and getting to know my colleagues and the things we’ll be working on.

But for the time being I have to say that my first impression has already been very positive and that I’m enthusiastically looking at the future and what it’s going to be next. Surely it will be a very different experience compared to what I was used to, but in a way that’s precisely what I was looking for, and so that’s why I feel very optimistic and motivated about it.

Also, and besides work related stuff, being a new resident in the UK also means that I had to spend some time doing some additional things, such as creating a bank account, getting a UK-based SIM card and starting to look for a place that should become our permanent residence in 2-3 months time, once my wife and my children move as well to the UK (they’re still in Spain), hopefully before Easter. Fortunately, being a EU citizen simplifies a lot the whole thing of coming here to work, since I don’t need any VISA or the like. Just my Spanish ID and/or Passport are more than enough.

Anyway, I’ve just arrived in the UK on Monday and started working on Tuesday (yeah, I love having big margins) so still much to do left, but I’m already on my way so it’s a matter of time that we are settled here, and that we start living the “English adventure” all together again.

But in the meanwhile we’ll have to live with me visiting them every 2-3 weeks and the typical audio/video conference tools.

Can’t wait!

Moving On

Published November 19, 2012 / by mario / 14 Comments on Moving On

Two months ago it was an important day for me. It was the day that I finally told my partners at Igalia that I’d be leaving the company, after almost seven years. It hasn’t been an easy decision to make and so this hasn’t been an easy post to write either… yet one I’d like to write anyway.

But let’s push the rewind button for a moment now…

I’ve joined Igalia on November 2005, initially as a trainee while I was finishing my studies of Computer Science Engineering, and got quickly hooked by the company, the people that was part of it back then and the kind of project they were trying to build.

Igalia as a company is one of a kind, you can be sure of it, and I felt enormously lucky for having found it and having been granted with the privilege of being part of it. And best of all… it happened to have its main office in my own town! I just couldn’t believe it…

I even still recall now how many mates of mine in the Faculty were telling me, back in the days of the University, things like “that Linux thing you like so much is not worth it”, “forget about Open Source, you won’t ever make a living out of it” or even “well, maybe there’s a chance for you to work on that, but it’s definitely not going to happen here”.

I have to say I never believed much in any of those statements (well, maybe a bit the last one), but I swear I couldn’t have ever imagined that I’d be able to prove all of them wrong at the same time without even having to move out of the country, let alone the town.

It was too good to be true. But it was true and real… it definitely was real.

And turns out that what started as a 3-month internship in late 2005, became my first permanent job (and the only one so far) when I became a regular employee on February 2006, to finally become a partner of the company in 2010. Not bad.

I worked on many different projects across all these years, from system administration tasks and pure web development to application development over the GNOME and the Maemo platforms, to end up working on what it probably became the most challenging and interesting thing I’ve ever worked on: the WebKit project and, more specifically, the WebKitGTK+ port.

Additionally, I also met a lot of awesome people while there, both inside Igalia and outside (e.g. at conferences), in some cases becoming closer to “actual friends” than to “just work mates”, which is yet another great thing I will always feel lucky and grateful for.

In other words, I grew up there, both at the professional and the personal level, and I feel endlessly grateful for that. I’m sure my life wouldn’t have been the same now without that stage in my life, so I don’t have more than good words and thoughts about it.

However, and even if I still think Igalia is an awesome place to be part of, I’ve been lately feeling like I needed a change, to try something new, to move on…

I’m not sure about how much that could be related to the fact that my life is now quite different than seven years ago, and that perhaps my priorities could have shifted now I have my own family, but the fact is that at some point I very clearly saw that I would need to try something else, to change some things and patterns in my life, kind of a “fresh restart”.

And after so much thinking, I suddenly realized such a change couldn’t ever happen if I stayed in Igalia, since it would probably require that the company stopped being as it is in some ways. And that’s something I don’t think I should ask for “just because I felt that way”. That would be too selfish, don’t you think?

So I made the only decision I thought it made sense: to quit.

Sure it was hard, and even tough at some moments, to make such a decision. But once I made it for real, I have to say I felt very well about it, as if everything was making sense once again, as if the puzzle I was trying so hardly to solve was finally complete.

In any case, this doesn’t mean I want to work on something completely different either. In my lollipop world, my plan is to keep working around the Open Source world as much as possible, hopefully also around GNOME and WebKit, the two platforms I learned to love during these years. Well, actually for the case of GNOME I must say it’s a platform I already loved before joining Igalia, but I can’t deny that being part of that company played a major role to help me get more involved on it, so that’s why I think it deserves some credit.

Fortunately, I’m a lucky guy and I can already say that, just two months after making that decision and starting looking for new opportunities, I already found a job where I think I’ll be able to keep working in what I love (see paragraph above) while, at the same time, being also able to try something different and new compared to what I’ve done so far.

Additionally, I think this new job will also help me find answers to the questions that have been crossing my mind lately, let alone learning new stuff… I’m sure I’ll learn a lot of new stuff as well, something I already can’t wait for.

Anyway, I won’t start working there until January, so let’s go step by step.

First I needed to write this post to tell the world about my new situation (check). Second, I need to use these “two months in nowhere” to re-organize my life and arrange many different things related to the short-term (work in progress). Finally, I’ll start that new stage in my life and finally announce here where I’m going to, something I’d rather keep for another post.

Surely I know all this will mean big challenges for both me and my family (we’ll be moving abroad, to begin with), but those are challenges we’re willing to face and can’t wait for. In a way, I kind of feel like I did 7 years ago right before joining Igalia, when I just finished my studies at the University and still had no clue what I was going to do with my life. Back then, the future looked uncertain and full of opportunities at the same time, and if I now look backwards I can clearly see it was even better than what I’ve would ever have expected. Really.

And you know what? That’s exactly how I feel right now, and that’s a feeling I’m liking a lot. And believe me, I’m pretty much aware that my current situation (married and with two children) is quite different compared to 7 years ago. Yet it doesn’t scare me well enough not to feel thrilled about this new stage in our life, a stage I’m already eager to deal with.

So it’s time to move on. I won’t ever forget the great time I had at Igalia, all the people I met here and all the good things that being part of such an special company brought to my life, but I also think now it’s time to look forward and focus on the future.

I have the most amazing family in the world and we’ll be together on this through thick and thin, so I’m already sure we’ll be fine. It’s just a matter of time we’ll get there, so what else can I ask for?

Just wish us good luck. That should do the rest :)

WebKitGTK+ 1.10 is almost here!

Published September 14, 2012 / by mario / 4 Comments on WebKitGTK+ 1.10 is almost here!

As you might already know, the new and shiny 3.6 release of the GNOME desktop is right around the corner, and so it’s the next release of WebKitGTK+, the port of the WebKit web rendering engine to the GTK+ platform.

And it turns out that such a release is going to be a very special one for us, members of the WebKit team at Igalia, since it comes with some very interesting features, like those I already mentioned in the talk I gave during the past GUADEC, mainly:

Beta version of the WebKit2GTK+ API
Support for Accelerated Compositing
WebGL enabled by default
Support for HTML5 Fullscreen and WebAudio
Multimedia layer ported to GStreamer 0.11
Support for the Low-Level Interpreter in JavaScriptCore

From all those, I’m specially happy because we will be finally releasing the very first beta version of the new WebKit2GTK+ API, based in the multi-process architecture of WebKit2, as well as providing support for Accelerated Compositing and WebGL.

This new WebKit2GTK+ API, as you perhaps already know, will allow applications gain the split process model of WebKit2 out-of-the box, which is awesome. Xan already mentioned some of the advantages of it becoming beta for GNOME 3.6 in his last post this week, being my favorite ones the “increased responsiveness and stability” (quoting Xan) that will come with it, as well as the fact that it will be not only powerful enough to port old applications and write new ones, but also simpler and easier to use (we are putting a lot of effort on this).

And honestly, I think we are doing pretty well in that regard, even though there’s still a lot of work to do before we can release an stable version of this new API (due for WebKitGTK+ 2.0, to be released with GNOME 3.8), which will also mean the very first version of Epiphany that will be using WebKit2 by default.

With regard to Accelerated Compositing and WebGL, I’d just like to mention that having them supported in WebKitGTK+ from now on is great because it means you will be able to render visually stunning web content in your browser of choice (epiphany, huh?), as well as enjoy more subtle improvements such as smoother animations or increased responsivenes while browsing. You can visit this post by my mate Martin for more details on this topic.

Anyway, all these are very nice words and all that, but sometimes it’s not that easy to properly understand just with words what exactly those things will actually mean for users, so I decided to spend some time today polishing a bit the videos I used as demos in my talk during GUADEC, and link them from here, so everyone can easily watch them now.

Hope you enjoy watching them as much as I did making them:

WebKitGTK+: WebGL and Accelerated Compositing

WebKit2GTK+: The UI and the Web process

WebKit2GTK+: The Plugin process

mariospr.org

Published August 24, 2012 / by mario / 1 Comment on mariospr.org

I’ve been thinking for some time already of moving my six years old blog to a more personal domain, and turns out I did it last week moved by the fact that I already had to mess with web development, hostings and domains for a while again, in order to assist my brother-in-law with the set up of a new local business.

So, I finally went after some (quick) thinking for this mariospr.org domain which I think it kinda makes sense considering I’m using mariospr in many places out there already.

One consequence of this change is that I hope I will write more often in my blog from now on, and not only about work-related stuff but also about some more personal things, experiences and the like. Still, I plan to keep using the “Planet [Name]” categories to control what I send to the planets I’m registered in, so if you want to keep track of those other kind of posts, make sure you visit or subscribe to the blog’s feed URL.

By the way, I would like to make the most of this post to say thank you to Lucas Rocha, who have helped me quite a lot to set up this blog without even knowing it, simply by being a good “inspiration” for some decisions I needed to make during the migration process, such as the domain name I finally chose and the font family for the text, for instance.

I even tried the sleek WordPress theme Lucas created, although in this case I ended up making my own slight modifications to the -also beautiful- TwentyEleven theme, which I will publish as well at some point when the blog has really settled down.

Last, thanks also to my mates in Igalia who have already set up a redirect rule from the old address at blogs.igalia.com to the new domain, and of course to the people who helped update the feed link in the different planets my blog is in, since that means that everything should be already working as expected, even if you’re still using the old URLs.

GUADEC, WebKit and bikes

Published July 20, 2012 / by mario

It seems this year GUADEC is going to be pretty close to my place and so I will surely attend, but this time I won’t go by plane but by bike, which since some months ago has become my main vehicle for moving around the beautiful city where I live in: A Coruña.

Also, besides hanging around the venue and trying to help as much as possible as the local I am, I’ll be talking about WebKitGTK+ in the afternoon on Thursday 26th, so feel free to come round the room if you feel curious about the current status of the whole thing and the current plans for the short and medium term, which are mostly focused around WebKit2 and the roadmap we’re already following.

You probably already read some news related to this coming from my mates in the Igalia WebKit team, (like the improvements in Accelerated Compositing or the migration of our handsome browser Epiphany to using WebKit2), yet I will try to deliver an interesting talk to y’all. I just hope I’ll be able to do it (but please forgive me if I don’t).

So that’s it. As usual, just feel free to talk me if you see me around if you want. I’ll basically be around the venue most of the time during GUADEC, and will attend a11y and WebKitGTK+ BoFs on the 30th and 31st, so I’d say it will be pretty easy to find me.

WebKit Watcher 0.2 released

Published June 1, 2012 / by mario / 2 Comments on WebKit Watcher 0.2 released

Some time ago I wrote a (very!) small android application to scratch an itch I had: being able to easily check the status of WebKit buildbots right from my phone, which turned out to be quite useful in some scenarios.

And no, I’m not the kind of guy that does the “Land Patch / Goes Home” thing, but sometimes you see yourself in the situation of having to leave your workplace before than expected and after having committed certain kind of patches a while ago. And in such a situation WebKit Watcher has proved to be a helpful tool many times, at least to me.

So, what’s new in this 0.2 release? Not many bells and whistles, to be honest, I just scratched a couple of small itches more I felt today, while going back to working as gardener for the WebKitGTK+ port. Basically:

Replaced the main view showing only the core bots with a (static) list of the different platforms, so you know can now check the results for all the bots of a specific platform. Selecting a platform will take you to the typical view showing the results for every build bot associated to it.
Added possibility of checking the results for all the bots in a single list view (‘All’ item in the main view), as previously only those “marked” as ‘core bots’ were shown in the application. Now you can check the results of all the 43 bots, in case you want to.
Removed unnecessary permission CHANGE_NETWORK_STATE, as I personally hate apps asking me for more permissions than those strictly needed. I’m still wondering why I added that in the first place…

And now, some screenshots that will make your eyes bleed because of the ‘design’ of this app:

I told you it was so beautiful!

By the way, you can grab it and install this app through several ways:

Google Play: just look for “webkit”
The .APK file to install by copying to the device: grab it from here
Source code: get it from its git repository

Back to FOSDEM

Published February 1, 2012 / by mario

So it seems I’m going to FOSDEM this year (yay!), together with a bunch of other Igalians who will be attending as well, coming from different places from across the globe (well, mainly from Europe this time).

I know some people will probably disagree with me on this, but for me FOSDEM is one of the greatest events of this kind, and so I’m quite happy to go there this time, specially after not being able to attend last year due to some unexpected (and unavoidable) personal matters.

Opposite to other occasions, this time I’ll be there not only as an attendant but also as an speaker, talking about WebKitGTK+, its status and the roadmap of the project towards WebKit2 (the split process model “flavour” of WebKit), together with my mate Philippe, on Sunday afternoon. Thus, for the first time ever, nobody will be able to accuse me of going there just because of the beer event, which wouldn’t be true anyway.

For the impatient ones, the talk will be mainly about reporting on the work done during the last months in “WebKitGTK+ land“, as well as on the stuff that is already planned for the upcoming releases. Good examples of those would be, for instance, the ongoing effort to add support for Accelerated Compositing, or just the new features related to WebKit2GTK+ such as, of course, the solution for enabling accessibility support there. Ah! And of course, we’ll try to run some demos there too… fingers crossed!

Besides, I’m of course looking forward to meeting some people I haven’t seen for a while now (haven’t attended to the latest Desktop Summit either, due to very good reasons too), so if you see me around and want to chat and/or meet for a while, just let me know. I must look shy, but it’s usually a matter of minutes (seconds?) for my shyness to go away…

So that’s it. Just a final line to say “thanks” to my company for fully sponsoring this thing.

See you in Brussels!

Accessibility support in WebKit2GTK+

Published January 27, 2012 / by mario / 1 Comment on Accessibility support in WebKit2GTK+

As Piñeiro already mentioned in some posts, last week a bunch of hackers attended the ATK/AT-SPI Hackfest 2012 here at the Igalia offices, in the lovely city of Coruña.

As the guy working on accessibility support for WebKitGTK+, I attended the hackfest to join some other great people representing different projects, such as Mozilla, Orca, AT-SPI, ATK, GTK+ and Qt. So, apart from helping with some “local” organizational details of the hackfest and taking some pictures, I spent some time hacking in WebKitGTK+‘s accessibility code and participating in some discussions.

And from that dedication I managed to achieve some interesting things too, being my favorite ones a big refactoring of the a11y code in WebCore (so it’s now better organized and hence more readable and easy to hack on) and pushing my patch for enabling accessibility support in WebKit2GTK+, after going through a meticulous process of review (see the related WK bug), which started with the patch I wrote and attached back when attending to the WebKitGTK+ hackfest, as I mentioned in my previous entry in this blog.

Yeah, I know that some weeks have already passed since then and so perhaps you’re thinking this could have been done faster… but I’ve spent some weeks on holidays in Barcelona in December (pictures here!) and so I wouldn’t have much time before January to devote to this task. However, the patch got integrated faster than what I would expect when I proposed the first version of it, so I’m quite satisfied and happy anyway just by being able to announce this at this moment. Hope you share my joy :-)

So, what does this mean from the point of view of accessibility in GNOME? Well, that’s an easy question to answer: from now on, every browser that uses WebKit2GTK+ will be as much accessible as those using the previous version of WebKitGTK+, and this is definitely a good thing. Of course, I’m certain there will be bugs in this specific part that will need fixing (as it always happens), but for the time being this achievement means “yet another thing less” preventing us from pushing for upgrading some applications to switch to WebKit2GTK+, such as devhelp (some ongoing work already done, as my mate Carlos announced yesterday), yelp, liferea… and the mighty Epiphany browser, which is rocking more and more ech day that goes by.

Last, I’d like to share with you an screenshot showing this new stuff, but as I am a little bit tired of always using Minibrowser (that small browser we use for testing WebKit2), so I decided to try instead that new branch Carlos recently pushed for devhelp, so you could check that what I mentioned before is actually true.

So here you have it (along with a couple of additions done with Gimp):

As you can see, devhelp is running and Accerciser is showing the full hierarchy of accessible objects associated to the application, starting in the UI process (GTK+ world) and continuing in the Web process, where all the accessible objects from the WebKitGTK+ world are being exposed. As I explained in a previous post, the magic making possible the connection between the two process is done by means of the AtkSocket and the AtkPlug classes, also represented in the screenshot attached above.

So, that’s it.