Setting up an environment for cross-compiling WebKit2GTK+ for ARM

Of course, I know for a fact that many people use local recipes to cross-compile WebKit2GTK+ for ARM (or simply build in the target machine, which usually takes a looong time), but those are usually ad-hoc things and hard to reproduce environments locally (or at least hard for me) and, even worse, often bound to downstream projects, so I thought it would be nice to try to have something tested with upstream WebKit2GTK+ and publish it on trac.webkit.org,

So I spent some time working on this with the idea of producing some step-by-step instructions including how to create a reproducible environment from scratch and, after some inefficient flirting with a VM-based approach (which turned out to be insanely slow), I finally settled on creating a chroot + provisioning it with a simple bootstrap script + using a simple CMake Toolchain file, and that worked quite well for me.

In my fast desktop machine I can now get a full build of WebKit2GTK+ 2.14 (or trunk) in less than 1 hour, which is pretty much a productivity bump if you compare it to the approximately 18h that takes if I build it natively in the target ARM device I have :-)

Of course, I’ve referenced this documentation in trac.webkit.org, but if you want to skip that and go directly to it, I’m hosting it in a git repository here: github.com/mariospr/webkit2gtk-ARM.

Note that I’m not a CMake expert (nor even close) so the toolchain file is far from perfect, but it definitely does the job with both the 2.12.x and 2.14.x releases as well as with the trunk, so hopefully it will be useful as well for someone else out there.

Last, I want to thanks the organizers of this event for making it possible once again (and congrats to Igalia, which just turned 15 years old!) as well as to my employer for supporting me attending the hackfest, even if I could not make it in person this time.

It will be different for me this time, though, as now my main focus won’t be on accessibility (yet I’m happy to help with that, too), but on fixing a few issues related to the WebKit2GTK+ API layer that I found while working on our platform (Endless OS), mostly related to its implementation of accelerated compositing.

Besides that, I’m particularly curious about seeing how the hackfest looks like now that it has broaden its scope to include other web engines, and I’m also quite happy to know that I’ll be visiting my home town and meeting my old colleagues and friends from Igalia for a few days, once again.

Last, I’d like to thank my employer for sponsoring this trip, as well as Igalia for organizing this event, one more time.

See you in Coruña!

The following CMake snippet was not properly recognizing my 32bit chroot:

string(TOLOWER ${CMAKE_HOST_SYSTEM_PROCESSOR} LOWERCASE_CMAKE_HOST_SYSTEM_PROCESSOR)
if (CMAKE_COMPILER_IS_GNUCXX AND "${LOWERCASE_CMAKE_HOST_SYSTEM_PROCESSOR}" MATCHES "(i[3-6]86|x86)$")
    ADD_TARGET_PROPERTIES(WebCore COMPILE_FLAGS "-fno-tree-sra")
endif ()

After some investigation, I found out that CMAKE_HOST_SYSTEM_PROCESSOR relies on the output of uname to determine the type of the CPU, and this what I was getting if I ran it myself:

(debian32-chroot)mario:~ $ uname -a
Linux moucho 4.0.6-300.fc22.x86_64 #1 SMP Tue Jun 23 13:58:53 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux

Let’s avoid nasty comments about the stupid name of my machine (I’m sure everyone else uses clever names instead), and see what was there: x86_64.

That looked wrong to me, so I googled a bit to see what others did about this and, besides finding all sorts of crazy hacks around, I found that in my case the solution was pretty simple just because I am using schroot, a great tool that makes life easier when working with chrooted environments.

Because of that, all I would have to do would be to specify personality=linux32 in the configuration file for my chrooted environment and that’s it. Just by doing that and re-entering in the “jail”, the output would be much saner now:

(debian32-chroot)mario:~ $ uname -a
Linux moucho 4.0.6-300.fc22.x86_64 #1 SMP Tue Jun 23 13:58:53 UTC 2015
i686 i686 i686 GNU/Linux

And of course, WebKitGTK+ would now recognize and use the right CPU type in the snippet above and I could “relax” again while seeing WebKit building again.

Now, for extra reference, this is the content of my schroot configuration file:

$ cat /etc/schroot/chroot.d/00debian32-chroot
[debian32-chroot]
description=Debian-like chroot (32 bit) 
type=directory
directory=/schroot/debian32/
users=mario
groups=mario
root-users=mario
personality=linux32

That is all, hope somebody else will find this useful. It certainly saved my day!

As for me, I flew there to work mainly on accessibility related issues, making the most of the fact that both Joanie (Orca maintainer) and Piñeiro (ATK maintainer) would be there too, so it should be possible to make things happen faster, specially discussion-wise.

And turns out that, even if I feel like I could have achieved more than what I actually did (as usual), I believe we did quite well in the end: we discussed and clarified things that were blocking the mapping of new WAI-ARIA roles in WebKitGTK+, we got rid of a bunch of WebKit1-specific unit tests (Joanie converted them into nice layout tests that will be run by WebKit2GTK+ too), we got a few new roles in ATK to be able to better map things from the web world and and we fixed a couple of issues in the way too.

Of course, not everything has been rainbows and unicorns, as it seems that one of the patches I landed broke the inspector for WebKit2GTK+ (sorry Gustavo!). Fortunately, that one has been rolled out already and I hope I will be able to get back to it soon (next week?) to provide a better patch for that without causing any problem. Fingers crossed.

In the other hand, my mate Brian Holt joined us for three days too and, despite of being his first time in the hackfest, he got integrated pretty quickly with other hackers, teaming up to collaborate in the big boost that the network process & multiple web processes items have went through during the event. And not only that, he also managed to give a boost to his last patch to provide automatic memory leak detection in WebKitGTK+, which I’m sure it will be a great tool once it’s finished and integrated upstream.

Anyway, if you want more details on those topics, or anything else, please check out the blog posts that other hackers have been posting these days, specially Carlos’s blog post, which is quite extensive and detailed.

Of course, I would like to thank the main sponsors Igalia and the GNOME Foundation for making this thing happen again, and to my employer Samsung  for helping as well by paying our trips and accommodation, as well as the snacks and the coffee that helped us stay alive and get fatter during the hackfest.

Last, I would like to mention (in case anyone reading this wondered) that it has indeed felt a bit strange to go the city where I used to live in and stay in a hotel, not to mention going to the office where I used to work in and hang around it as a visitor. However, both my former city and my former colleagues somehow ensured that I felt as “at home” once again, and so I can’t do anything about it but feeling enormously grateful for that.

Thank you all, and see you next year!

And finally, I’m really happy to say that such a task is complete once and for all, meaning that now those functions should work as well or as bad on WebKit2GTK+ as they do in WebKitGTK+, so the weird behaviour described in bug 73433 is no longer an issue. You can check I’m not lying by just taking a look to the commit that removed both all trace of Pango and GAIL in the code, as well as and the one that removed the GAIL dependency from the build system. And if you want more detail, just feel free to check the whole dependency tree in WebKit’s bugzilla.

This task has been an interesting challenge for me indeed, and not only because it was one of the biggest accessibility related tasks I’ve worked on in WebKitGTK+ since late 2012 (so I needed to get my brain trained again on it), but also because reimplementing these functions forced me to dive into text editing and accessibility code in WebCore as I never did before. And it’s so cool to see how, despite of having to deal eventually with the frustrating feeling of hitting my head against a wall, at the end of the day it all resulted on a nice set of patches that do the work and help advance the state of the ATK based accessibility layer in WebKitGTK+ forward.

Anyway, even though that is probably the thing that motivated me to write this blog post, that was not the only thing that I did since GUADEC (which has been a blast, by the way):

On the personal side, I’ve spent two lovely weeks in Spain on holidays, which was the biggest period of time I’ve been outside of the UK since I arrived here, and had  an amazing time there just “doing nothing”(tm) but lying around on the beach and seeing the grass grow. And there is not much grass there anywhere, so you can imagine how stressful that life was… it was great.

In the other hand, on the professional side, I’d say that one of the other big things that happened to me lately was that I finally became accepted as a WebKit reviewer, meaning that now I can not only help breaking the Web, but also authorize others to do it so. And while agree that might be fun in some way, it probably would not be very cool, so forgive me if I try instead to do my best to help get exactly the opposite result: make things work better.

And truth to be told, this “upgrade” came just with perfect timing, since these days quite some work is being done in the accessibility layer for both the WebKitGTK+ and the WebKitEFL ports thanks also to my mates Denis Nomiyama, Anton Obzhirov, Brian Holt and Krzysztof Czech from Samsung, and that work would ideally need to be reviewed by someone familiar with the ATK/AT-SPI based accessibility stack. And while I’m still by far not the most knowledgeable person in the world when it comes to those topics, I believe I might have a fairly well knowledge about it anyway, so I assume (and hope) that my reviews will certainly add value and help with those specific pieces of work.

And as a nice plus, now I can finally “return the favour” to the only accessibility reviewer WebKit had until now (Chris Fleizach) by helping reviewing his patches as well, in a similar fashion to what he has been tirelessly doing for me for the last 3 years and a half. Yay!

To finish , I’d like to get back again to the original topic of this post and say a big “thank you” to everyone who helped me along the way with the removal of Pango and GAIL from the ATK specific code. Special thanks go to those who spend time performing the code reviews, as it’s the case of Martin, Gustavo and Chris. I wouldn’t be writing this post otherwise.

Thanks!

And the reason why it was not initially in my plans was mainly because I’ve been already through quite some changes during these past months year, and my family just came over to the UK two months ago. This means that, even I already arrived by the beginning of the year, we just started to settle here as a family a few weeks ago. So in that context, I didn’t feel like leaving them alone for one week already now, it definitely would look like a “wrong management of priorities” to me.

However, it turns out that my wife and kids won’t be here anyway during the first week of August and, on top of that, Samsung has been so kind to sponsor this trip just based on the simple fact that I’m part of the GNOME community. So, I certainly can no longer find a single reason not to go and spend 7 amazing days in Brno, meeting people that I normally see only in conferences (and this time that group of people will be bigger than ever, since my former mates from Igalia are now also included there), while attending to what it seems to be a very appealing event.

Also, I will try to make the most of the trip to do some work during the different hackfests and BoFs that are already planned, which special focus in the one about accessibility, of course. As a personal goal, I expect to have the chance to move forward some work I’ve been doing lately in the WebKitGTK+ a11y world, such as getting rid of the nasty dependency on Pango/Gail we still have there, something I’ve been already working on for some time now, and which I expect it will be fixed soon, hopefully before GUADEC, although time will tell.

Once that it’s fixed, WebKit2GTK+ based apps should recover the ability to properly expose text through the atk_text_get_text_*_offset() family of functions for different text boundaries, which means that ATs (e.g. the Orca screen reader)  will be able to properly allow again line-by-line navigation when in caret browsing mode. And, as you can imagine, this is quite a big problem these days, since WebKit2GTK+ that has become the default backend for some core apps such as the Epiphany browser with the GNOME 3.8 release, so fixing this is like a high priority now, I’d say.

Anyway, I’m starting to write too much (as usual) for what it was going to be a short “I’m going to GUADEC” blog post, so I will stop right now, although not without first thanking Samsung for sponsoring this my first trip to the Czech Republic.

See you all in three weeks!

With regard to that, it has been a very interesting experience so far where I could meet new people I still haven’t had the chance to see in real life yet (e.g. my mates from other Samsung R&D centers or some guys from Apple I didn’t have the chance to meet in person before), as well as chat again with some friends and former mates that I haven’t seen for a while, such as Martin, Xan and Philippe from Igalia, Byungseon from LG, Nayan from Motorola or Gustavo from Collabora to mention some of them. It’s strange, and at the same time wonderful, how easily you can catch up on conversations with people that you barely see once a year (or even less) and mainly in conferences, and definitely one of my favourite parts of attending these kind of events, to be honest.

Also, from a less social point of view, I have to say I found very interesting the sessions I’ve attended so far, specially the one about “managing the differences between ports”, although the one about “build systems” was quite interesting too. Not sure how far we are yet in the WebKitGTK+ port from realistically switching to some kind of commonly agreed build system (cmake?), but at least it’s a good start to agree on the fact that it would be an interesting move and now that some people pushing for it.

My only regret about this first day is that I missed Hyatt’s talk about pagination due to some health issues I’m experimenting while in California, mostly due to the extremely hot and dry weather (anything over 25 Celsius is “unbearable hot” for me), which is causing me a little bit of cough, sore throat and fever, all well mixed with the jet lag to make it a perfect “welcome pack” to the meeting. Fortunately, I got some “interesting” medicines that seem to have relieved a bit the pain and I could attend the rest of the sessions without much trouble, other than some occasional coughing. Not bad.

By the way, for those of you who were not lucky enough to attend the meeting but are anyway interested in the topics being discussed here, make sure you check the main TRAC page for the meeting, where you can also find transcripts for most of the sessions.

As for today, some more sessions will take place as well as a couple of hackathons so I expect it to be very interesting as well. Also I hope I can find some time too to work a bit on my patches to remove the nasty dependency on pango we have in WebKitGTK+ accessibility code, which is preventing us to have proper caret navigation in WebKit2GTK+ based browsers, as well as to discuss possible ways in which our lab could collaborate more actively upstream. Seems a promising day already!

Last (but not least), and in a completely unrelated and super-off-topic way, I would like to tell the world that I’m extremely happy for the fact that next week will be the end of my “lonely existence in the UK”, finally. After 4 months of living away from my family with just some flash trips from Friday to Sunday (every 2 weeks), I’m once and for all travelling on Thursday to my home town with a one way plane ticket to do some final arrangements, put everything (family included!) in the car and travel to Santander, where we’ll be taking a ferry that will take us to the Portsmouth (southern coast of England), from where we will just drive to the UK in order to start our new life, all together again.

It has been quite hard for us to live this way for so long, but I think in the end we managed to handle the situation quite well, and now it seems all our efforts are already paying off because things seem to be finally fitting in the right places: we have a lovely house, we have a place in a nearby public school for my oldest kid to start on September, most of the needed paperwork seems to be done and we already moved all our stuff from Spain (lots of toys!), which is now waiting to be used in our new place.

I really can’t wait to live again in the noisy and chaotic atmosphere that two kids can so easily create around them. Even if that means it will probably drive me crazy every now and then and that I won’t sleep that well sometimes.

Yes. Even considering that.

In order to help them better understand this topic, I wrote a summary to have as reference that explains in my own words which the main pieces of the whole puzzle are, and how they relate to one another. In my experience, it’s not always easy to understand the big picture quickly, and I think this kind of documentation can be quite useful for anyone willing to contribute to accessibility in WebKitGTK+. At least it would have been useful for me when I started working on this. I only regret not having written it before, but better late than never, right?

So let’s begin then. I will start by talking about accessibility-only stuff, which are basically common to any accessible application based in GTK+. Then I will explain the bits specific to WebKitGTK+ and how they fit in the picture.

Accessibility in GTK+ applications

• Assistive Technologies (ATs)
• AT-SPI (Assistive Technology Service Provider Interface)
• ATK (Accessibility ToolKit)
• ATK <-> AT-SPI bridge
• GTK+
• GTK's Accessibility Implementation

Now let’s describe all those points, one by one:

Assistive Technologies (ATs):

ATs are applications whose main purpose is to facilitate access and/or interaction with certain bits of information interesting from an accessibility related point of view, exposed by other applications. This access/interaction can be primarily output based. For instance the Orca screen reader is an AT which provides access via text to speech and/or refreshable braille to on-screen information exposed by editors, browsers, mail agents and other applications.

Other ATs are primarily input based, allowing the user to interact with the exposed applications by executing certain actions over them (e.g. clicking on a exposed link), so it’s not just about “consuming” information. Normally, ATs are called the clients and the applications exposing information the servers, as in the end it’s actually an implementation of a typical client/server architecture.

AT-SPI (Assistive Technology Service Provider Interface):

Set of interfaces that Assistive Technologies (the clients) understand and use to inspect and interact with the accessible content exposed by applications in Linux environments. At some point, “someone” has to provide actual AT-SPI objects (linked together forming a AT-SPI hierarchy) implementing several of those interfaces (depending of the type of object) so ATs can “see them”.

This is the job of the AT-SPI registry, a daemon which takes care of maintaining a hierarchy of AT-SPI objects for every single accessible application in the system, in a centralized way, so ATs can interact with them. It is worth mentioning that the parent/children relationships in that hierarchy are modelled in terms of D-Bus, so different AT-SPI objects can belong to different processes.

ATK (Accessibility ToolKit):

The toolkit used by GTK+ applications to expose accessible representations of the toolkit’s objects, along with appropriate interfaces, on the side of the applications exposing content (the servers). This representation is an almost a 1:1 match with the objects and interfaces defined by AT-SPI (that is, almost).

The main difference when it comes to understanding its place in the puzzle is that AT-SPI is what clients (ATs) understand, and that is not process-bounded (see previous point). ATK, in contrast, is what servers implement to expose accessible information, and it is process-bounded. Thus the parent/children relationships in the ATK hierarchy are modelled by actual references (pointers) between objects living in the same process.

ATK <-> AT-SPI bridge:

The glue that makes sure there’s a mapping between the ATK hierarchy living in the server process and the AT-SPI hierarchy held by the AT-SPI registry. Such a bridge is implemented in terms of D-Bus too, as it needs to communicate with the registry whenever something needs to be updated there, as well as when the server needs to react to external actions coming from ATs (e.g. perform the default action for an object).

GTK+:

The widgets toolkit normally used by applications embedding WebKitGTK+. Explaining what GTK+ is beyond the scope of this post, so I will assume you already know what it is.

GTK’s Accessibility Implementation:

Provides ATK objects implementing different ATK interfaces for every widget from the GTK+ library, and uses the ATK <-> AT-SPI bridge to communicate with the AT-SPI registry. This means that if you use standard GTK+ widgets only, your application will be accessible out-of-the-box. On the contrary, should you use custom widgets, you’ll probably have to write custom ATK objects implementing the proper ATK interfaces to make them accessible too.

So that’s all so far, when it comes to GTK+ applications. Check the following diagram for a more detailed look at all these hierarchies for a hypothetical GTK+ application exposing information and a screen reader accessing it:

As you can see, there’s an ATK tree matching the GTK+ hierarchy, and another AT-SPI tree matching the ATK one. Finally, the screen reader accesses the information through that AT-SPI tree, as explained above.

Accessibility in WebKitGTK+

• WebCore's Accessibility Objects
• WebKitGTK+ (ATK) wrappers
• WebKit2GTK+ specific details

In order to clarify it a bit more before explaining each point, let’s just say that  you’ll have to look in the dashed box named WebCore accessibility world, where the hierarchy on the left (red & orange) represent the WebCore Accessibility objects, while the one on the right (the green one) represents the WebKitGTK+ ATK wrappers.

With this in mind, let’s examine these three points in more depth:

WebCore Accessibility objects:

Similar to GTK’s Accessibility Implementation, WebCore’s accessibility objects are the implementation of an independent hierarchy exposing accessibility related information for objects present in a web page. As the mission of accessibility in WebKit is to expose information to users that are normally being rendered in the screen (as well as some other information that might be hidden to regular users), there is a tight relationship between this hierarchy and other ones in WebKit, such as the DOM tree and the Render Objects tree.

This layer is meant to be platform-agnostic, so you won’t find much WebKitGTK+ specific stuff here. Instead, you will find the implementation of the accessibility related specifications published by the W3C‘s Web Accessibility Initiative (WAI), such as WAI-ARIA.

WebKitGTK+ ATK wrappers:

An ATK-based implementation of an accessibility hierarchy where every ATK object will take care of wrapping the proper accessibility object from WebCore, as well as implementing the proper ATK interfaces depending on the situation (e.g. the role of the WebCore accessibility object, some properties coming from the associated Render Object…).

The ATK hierarchy created here is connected with the ATK hierarchy from the embedding application (normally a GTK+ app) by setting the root ATK object in this tree (normally representing DOM’s root element) as the child of the leaf ATK object in the tree coming from the embedding application (normally the GtkWidget containing the WebView).

As is the case with any other regular GTK+ application, this ATK hierarchy will finally be seen by ATs thanks to the translation that the ATK <-> AT-SPI bridge will do for us, making the whole ATK tree from the WebKitGTK+ based application (from the top level GTK+ window down to the deepest accessibility object inside WebCore) available to the AT-SPI registry by means of D-Bus.

WebKit2GTK+ specific details:

I already talked about this in previous posts, so I will focus here just on commenting the main difference compared to the generic case for WebKitGTK+ described earlier (see previous diagram above):

WebKit2GTK+ implements a split-process model, where the  high level API belongs to one process (the UI process) while the core logic of the web engine lives in another one (the Web process).

From an accessibility point of view, this means that the full hierarchy of ATK objects we had before is also split in two parts: some accessibility objects are now in the UI process and the rest of them will be in the Web process.

To be more specific, we’ll find the following objects in each process:

• UI Process: basically, the ATK object associated with the GtkWidget provided by WebKitGTK+, that is, the WebView.
• Web Process: the whole hierarchy of ATK objects wrapping the accessibility objects from WebCore.

Thus, since ATs just  understand AT-SPI, they will see The Right Thing (tm) as in the previous case where we have one single process. See the following diagram for a more visual explanation of this:

Wrapping up

At least I hope this will be helpful for anyone willing to contribute to accessibility, either in WebKitGTK+ or in a more general way.

After all, most of the stuff I talked about here applies to  every accesible GTK+ application: Assistive Technologies (ATs), AT-SPI, ATK, the ATK <-> AT-SPI bridge…

Last, I would like to thanks Joanmarie Diggs from Igalia for her help with this blog post. One certainly feels more confident writing a long article like this one about a very specific topic when you have one of the most experienced persons on the matter reviewing it!

And it turns out that such a release is going to be a very special one for us, members of the WebKit team at Igalia,  since it comes with some very interesting features, like those I already mentioned in the talk I gave during the past GUADEC, mainly:

• Beta version of the WebKit2GTK+ API
• Support for Accelerated Compositing
• WebGL enabled by default
• Support for HTML5 Fullscreen and WebAudio
• Multimedia layer ported to GStreamer 0.11
• Support for the Low-Level Interpreter in JavaScriptCore

This new WebKit2GTK+ API, as you perhaps already know, will allow applications gain the split process model of WebKit2 out-of-the box, which is awesome. Xan already mentioned  some of the advantages of it becoming beta for GNOME 3.6 in his last post this week, being my favorite ones the “increased responsiveness and stability” (quoting Xan) that will come with it, as well as the fact that it will be not only powerful enough to port old applications and write new ones, but also simpler and easier to use (we are putting a lot of effort on this).

And honestly, I think we are doing pretty well in that regard, even though there’s still a lot of work to do before we can release an stable version of this new API (due for WebKitGTK+ 2.0,  to be released with GNOME 3.8), which will also mean the very first version of Epiphany that will be using WebKit2 by default.

With regard to Accelerated Compositing and WebGL, I’d just like to mention that having them supported in WebKitGTK+ from now on is great because it means you will be able to render visually stunning web content in your browser of choice (epiphany, huh?), as well as enjoy more subtle improvements such as smoother animations or increased responsivenes while browsing. You can visit this post by my mate Martin for more details on this topic.

Anyway, all these are very nice words and all that, but sometimes it’s not that easy to properly understand just with words what exactly those things will actually mean for users, so I decided to spend some time today polishing a bit the videos I used as demos in my talk during GUADEC, and link them from here, so everyone can easily watch them now.

Hope you enjoy watching them as much as I did making them:

WebKitGTK+: WebGL and Accelerated Compositing

WebKit2GTK+: The UI and the Web process

WebKit2GTK+: The Plugin process

Also, besides hanging around the venue and trying to help as much as possible as the local I am, I’ll be talking about WebKitGTK+ in the afternoon on Thursday 26th, so feel free to come round the room if you feel curious about the current status of the whole thing and the current plans for the short and medium term, which are mostly focused around WebKit2 and the roadmap we’re already following.

You probably already read some news related to this coming from my mates in the Igalia WebKit team, (like the improvements in Accelerated Compositing or the migration of our handsome browser Epiphany to using WebKit2), yet I will try to deliver an interesting talk to y’all. I just hope I’ll be able to do it (but please forgive me if I don’t).

So that’s it. As usual, just feel free to talk me if you see me around if you want. I’ll basically be around the venue most of the time during GUADEC, and will attend a11y and WebKitGTK+ BoFs on the 30th and 31st, so I’d say it will be pretty easy to find me.