Category Archives: a11y

WebKitGTK+ Hackfest 2013: The Return of the Thing

Published December 13, 2013 / by mario

As many other WebKitGTK+ hackers (30 in total), I flew last Saturday to A Coruña to attend the 5th edition of the WebKitGTK+ Hackfest, hosted once again by Igalia at their premises and where people from several different affiliations gathered together to try to give our beloved port a boost.

As for me, I flew there to work mainly on accessibility related issues, making the most of the fact that both Joanie (Orca maintainer) and Piñeiro (ATK maintainer) would be there too, so it should be possible to make things happen faster, specially discussion-wise.

And turns out that, even if I feel like I could have achieved more than what I actually did (as usual), I believe we did quite well in the end: we discussed and clarified things that were blocking the mapping of new WAI-ARIA roles in WebKitGTK+, we got rid of a bunch of WebKit1-specific unit tests (Joanie converted them into nice layout tests that will be run by WebKit2GTK+ too), we got a few new roles in ATK to be able to better map things from the web world and and we fixed a couple of issues in the way too.

Of course, not everything has been rainbows and unicorns, as it seems that one of the patches I landed broke the inspector for WebKit2GTK+ (sorry Gustavo!). Fortunately, that one has been rolled out already and I hope I will be able to get back to it soon (next week?) to provide a better patch for that without causing any problem. Fingers crossed.

In the other hand, my mate Brian Holt joined us for three days too and, despite of being his first time in the hackfest, he got integrated pretty quickly with other hackers, teaming up to collaborate in the big boost that the network process & multiple web processes items have went through during the event. And not only that, he also managed to give a boost to his last patch to provide automatic memory leak detection in WebKitGTK+, which I’m sure it will be a great tool once it’s finished and integrated upstream.

Anyway, if you want more details on those topics, or anything else, please check out the blog posts that other hackers have been posting these days, specially Carlos’s blog post, which is quite extensive and detailed.

Of course, I would like to thank the main sponsors Igalia and the GNOME Foundation for making this thing happen again, and to my employer Samsung for helping as well by paying our trips and accommodation, as well as the snacks and the coffee that helped us stay alive and get fatter during the hackfest.

Last, I would like to mention (in case anyone reading this wondered) that it has indeed felt a bit strange to go the city where I used to live in and stay in a hotel, not to mention going to the office where I used to work in and hang around it as a visitor. However, both my former city and my former colleagues somehow ensured that I felt as “at home” once again, and so I can’t do anything about it but feeling enormously grateful for that.

Thank you all, and see you next year!

Goodbye Pango! Goodbye GAIL!

Published September 13, 2013 / by mario / 2 Comments on Goodbye Pango! Goodbye GAIL!

As I mentioned in my previous post before GUADEC, I’ve been putting some effort lately on trying to improve the accessibility layer of WebKitGTK+, as part of my work here at Samsung, and one of the main things I’ve worked on was the removal of the dependency we had on Pango and GAIL to implement the atk_text_get_text_*_offset() family of functions for the different text boundaries.

And finally, I’m really happy to say that such a task is complete once and for all, meaning that now those functions should work as well or as bad on WebKit2GTK+ as they do in WebKitGTK+, so the weird behaviour described in bug 73433 is no longer an issue. You can check I’m not lying by just taking a look to the commit that removed both all trace of Pango and GAIL in the code, as well as and the one that removed the GAIL dependency from the build system. And if you want more detail, just feel free to check the whole dependency tree in WebKit’s bugzilla.

This task has been an interesting challenge for me indeed, and not only because it was one of the biggest accessibility related tasks I’ve worked on in WebKitGTK+ since late 2012 (so I needed to get my brain trained again on it), but also because reimplementing these functions forced me to dive into text editing and accessibility code in WebCore as I never did before. And it’s so cool to see how, despite of having to deal eventually with the frustrating feeling of hitting my head against a wall, at the end of the day it all resulted on a nice set of patches that do the work and help advance the state of the ATK based accessibility layer in WebKitGTK+ forward.

Anyway, even though that is probably the thing that motivated me to write this blog post, that was not the only thing that I did since GUADEC (which has been a blast, by the way):

On the personal side, I’ve spent two lovely weeks in Spain on holidays, which was the biggest period of time I’ve been outside of the UK since I arrived here, and had an amazing time there just “doing nothing”(tm) but lying around on the beach and seeing the grass grow. And there is not much grass there anywhere, so you can imagine how stressful that life was… it was great.

In the other hand, on the professional side, I’d say that one of the other big things that happened to me lately was that I finally became accepted as a WebKit reviewer, meaning that now I can not only help breaking the Web, but also authorize others to do it so. And while agree that might be fun in some way, it probably would not be very cool, so forgive me if I try instead to do my best to help get exactly the opposite result: make things work better.

And truth to be told, this “upgrade” came just with perfect timing, since these days quite some work is being done in the accessibility layer for both the WebKitGTK+ and the WebKitEFL ports thanks also to my mates Denis Nomiyama, Anton Obzhirov, Brian Holt and Krzysztof Czech from Samsung, and that work would ideally need to be reviewed by someone familiar with the ATK/AT-SPI based accessibility stack. And while I’m still by far not the most knowledgeable person in the world when it comes to those topics, I believe I might have a fairly well knowledge about it anyway, so I assume (and hope) that my reviews will certainly add value and help with those specific pieces of work.

And as a nice plus, now I can finally “return the favour” to the only accessibility reviewer WebKit had until now (Chris Fleizach) by helping reviewing his patches as well, in a similar fashion to what he has been tirelessly doing for me for the last 3 years and a half. Yay!

To finish , I’d like to get back again to the original topic of this post and say a big “thank you” to everyone who helped me along the way with the removal of Pango and GAIL from the ATK specific code. Special thanks go to those who spend time performing the code reviews, as it’s the case of Martin, Gustavo and Chris. I wouldn’t be writing this post otherwise.

Thanks!

I’m going to GUADEC!

Published July 12, 2013 / by mario / 1 Comment on I’m going to GUADEC!

One year again GUADEC is approaching and, also again, I’m very happy to say that I’ll be there as well this time, even if I have to recognize it was not on my plans for this year, at least not initially.

And the reason why it was not initially in my plans was mainly because I’ve been already through quite some changes during these past months year, and my family just came over to the UK two months ago. This means that, even I already arrived by the beginning of the year, we just started to settle here as a family a few weeks ago. So in that context, I didn’t feel like leaving them alone for one week already now, it definitely would look like a “wrong management of priorities” to me.

However, it turns out that my wife and kids won’t be here anyway during the first week of August and, on top of that, Samsung has been so kind to sponsor this trip just based on the simple fact that I’m part of the GNOME community. So, I certainly can no longer find a single reason not to go and spend 7 amazing days in Brno, meeting people that I normally see only in conferences (and this time that group of people will be bigger than ever, since my former mates from Igalia are now also included there), while attending to what it seems to be a very appealing event.

Also, I will try to make the most of the trip to do some work during the different hackfests and BoFs that are already planned, which special focus in the one about accessibility, of course. As a personal goal, I expect to have the chance to move forward some work I’ve been doing lately in the WebKitGTK+ a11y world, such as getting rid of the nasty dependency on Pango/Gail we still have there, something I’ve been already working on for some time now, and which I expect it will be fixed soon, hopefully before GUADEC, although time will tell.

Once that it’s fixed, WebKit2GTK+ based apps should recover the ability to properly expose text through the atk_text_get_text_*_offset() family of functions for different text boundaries, which means that ATs (e.g. the Orca screen reader) will be able to properly allow again line-by-line navigation when in caret browsing mode. And, as you can imagine, this is quite a big problem these days, since WebKit2GTK+ that has become the default backend for some core apps such as the Epiphany browser with the GNOME 3.8 release, so fixing this is like a high priority now, I’d say.

Anyway, I’m starting to write too much (as usual) for what it was going to be a short “I’m going to GUADEC” blog post, so I will stop right now, although not without first thanking Samsung for sponsoring this my first trip to the Czech Republic.

See you all in three weeks!

WebKit Contributors Meeting 2013

Published May 3, 2013 / by mario / 2 Comments on WebKit Contributors Meeting 2013

It turns out I’m writing this post at 6:00 AM in the morning from a hotel instead of doing it at a more reasonable time from my comfy home or a nice cafeteria. That’s already quite a new thing by itself, and the reason for that is not that I became crazy or something, but the fact that I’m completely jet-lagged in California right now in order to attend my second WebKit Contributors Meeting (my first time was in 2011), this time as part of the Samsung team in the UK R&D center, together with my mate Anton Obzhirov.

With regard to that, it has been a very interesting experience so far where I could meet new people I still haven’t had the chance to see in real life yet (e.g. my mates from other Samsung R&D centers or some guys from Apple I didn’t have the chance to meet in person before), as well as chat again with some friends and former mates that I haven’t seen for a while, such as Martin, Xan and Philippe from Igalia, Byungseon from LG, Nayan from Motorola or Gustavo from Collabora to mention some of them. It’s strange, and at the same time wonderful, how easily you can catch up on conversations with people that you barely see once a year (or even less) and mainly in conferences, and definitely one of my favourite parts of attending these kind of events, to be honest.

Also, from a less social point of view, I have to say I found very interesting the sessions I’ve attended so far, specially the one about “managing the differences between ports”, although the one about “build systems” was quite interesting too. Not sure how far we are yet in the WebKitGTK+ port from realistically switching to some kind of commonly agreed build system (cmake?), but at least it’s a good start to agree on the fact that it would be an interesting move and now that some people pushing for it.

My only regret about this first day is that I missed Hyatt‘s talk about pagination due to some health issues I’m experimenting while in California, mostly due to the extremely hot and dry weather (anything over 25 Celsius is “unbearable hot” for me), which is causing me a little bit of cough, sore throat and fever, all well mixed with the jet lag to make it a perfect “welcome pack” to the meeting. Fortunately, I got some “interesting” medicines that seem to have relieved a bit the pain and I could attend the rest of the sessions without much trouble, other than some occasional coughing. Not bad.

By the way, for those of you who were not lucky enough to attend the meeting but are anyway interested in the topics being discussed here, make sure you check the main TRAC page for the meeting, where you can also find transcripts for most of the sessions.

As for today, some more sessions will take place as well as a couple of hackathons so I expect it to be very interesting as well. Also I hope I can find some time too to work a bit on my patches to remove the nasty dependency on pango we have in WebKitGTK+ accessibility code, which is preventing us to have proper caret navigation in WebKit2GTK+ based browsers, as well as to discuss possible ways in which our lab could collaborate more actively upstream. Seems a promising day already!

Last (but not least), and in a completely unrelated and super-off-topic way, I would like to tell the world that I’m extremely happy for the fact that next week will be the end of my “lonely existence in the UK”, finally. After 4 months of living away from my family with just some flash trips from Friday to Sunday (every 2 weeks), I’m once and for all travelling on Thursday to my home town with a one way plane ticket to do some final arrangements, put everything (family included!) in the car and travel to Santander, where we’ll be taking a ferry that will take us to the Portsmouth (southern coast of England), from where we will just drive to the UK in order to start our new life, all together again.

It has been quite hard for us to live this way for so long, but I think in the end we managed to handle the situation quite well, and now it seems all our efforts are already paying off because things seem to be finally fitting in the right places: we have a lovely house, we have a place in a nearby public school for my oldest kid to start on September, most of the needed paperwork seems to be done and we already moved all our stuff from Spain (lots of toys!), which is now waiting to be used in our new place.

I really can’t wait to live again in the noisy and chaotic atmosphere that two kids can so easily create around them. Even if that means it will probably drive me crazy every now and then and that I won’t sleep that well sometimes.

Yes. Even considering that.

Accessibility in [WebKit]GTK+

Published February 3, 2013 / by mario / 8 Comments on Accessibility in [WebKit]GTK+

This past week I’ve spent some time explaining to my mates at Samsung the basics about how accessibility works and is implemented in WebKitGTK+. I realized, yet again, of how messy and confusing everything can be the first time you encounter these things. After all, WebKit is quite a complex project already and accessibility is not a simple matter either.

In order to help them better understand this topic, I wrote a summary to have as reference that explains in my own words which the main pieces of the whole puzzle are, and how they relate to one another. In my experience, it’s not always easy to understand the big picture quickly, and I think this kind of documentation can be quite useful for anyone willing to contribute to accessibility in WebKitGTK+. At least it would have been useful for me when I started working on this. I only regret not having written it before, but better late than never, right?

So let’s begin then. I will start by talking about accessibility-only stuff, which are basically common to any accessible application based in GTK+. Then I will explain the bits specific to WebKitGTK+ and how they fit in the picture.

Accessibility in GTK+ applications

The parts, or “actors”, involved in any GTK+ application from an accessibility point of view are:

Assistive Technologies (ATs)
AT-SPI (Assistive Technology Service Provider Interface)
ATK (Accessibility ToolKit)
ATK <-> AT-SPI bridge
GTK+
GTK’s Accessibility Implementation

Now let’s describe all those points, one by one:

Assistive Technologies (ATs):

ATs are applications whose main purpose is to facilitate access and/or interaction with certain bits of information interesting from an accessibility related point of view, exposed by other applications. This access/interaction can be primarily output based. For instance the Orca screen reader is an AT which provides access via text to speech and/or refreshable braille to on-screen information exposed by editors, browsers, mail agents and other applications.

Other ATs are primarily input based, allowing the user to interact with the exposed applications by executing certain actions over them (e.g. clicking on a exposed link), so it’s not just about “consuming” information. Normally, ATs are called the clients and the applications exposing information the servers, as in the end it’s actually an implementation of a typical client/server architecture.

AT-SPI (Assistive Technology Service Provider Interface):

Set of interfaces that Assistive Technologies (the clients) understand and use to inspect and interact with the accessible content exposed by applications in Linux environments. At some point, “someone” has to provide actual AT-SPI objects (linked together forming a AT-SPI hierarchy) implementing several of those interfaces (depending of the type of object) so ATs can “see them”.

This is the job of the AT-SPI registry, a daemon which takes care of maintaining a hierarchy of AT-SPI objects for every single accessible application in the system, in a centralized way, so ATs can interact with them. It is worth mentioning that the parent/children relationships in that hierarchy are modelled in terms of D-Bus, so different AT-SPI objects can belong to different processes.

ATK (Accessibility ToolKit):

The toolkit used by GTK+ applications to expose accessible representations of the toolkit’s objects, along with appropriate interfaces, on the side of the applications exposing content (the servers). This representation is an almost a 1:1 match with the objects and interfaces defined by AT-SPI (that is, almost).

The main difference when it comes to understanding its place in the puzzle is that AT-SPI is what clients (ATs) understand, and that is not process-bounded (see previous point). ATK, in contrast, is what servers implement to expose accessible information, and it is process-bounded. Thus the parent/children relationships in the ATK hierarchy are modelled by actual references (pointers) between objects living in the same process.

ATK <-> AT-SPI bridge:

The glue that makes sure there’s a mapping between the ATK hierarchy living in the server process and the AT-SPI hierarchy held by the AT-SPI registry. Such a bridge is implemented in terms of D-Bus too, as it needs to communicate with the registry whenever something needs to be updated there, as well as when the server needs to react to external actions coming from ATs (e.g. perform the default action for an object).

GTK+:

The widgets toolkit normally used by applications embedding WebKitGTK+. Explaining what GTK+ is beyond the scope of this post, so I will assume you already know what it is.

GTK’s Accessibility Implementation:

Provides ATK objects implementing different ATK interfaces for every widget from the GTK+ library, and uses the ATK <-> AT-SPI bridge to communicate with the AT-SPI registry. This means that if you use standard GTK+ widgets only, your application will be accessible out-of-the-box. On the contrary, should you use custom widgets, you’ll probably have to write custom ATK objects implementing the proper ATK interfaces to make them accessible too.

So that’s all so far, when it comes to GTK+ applications. Check the following diagram for a more detailed look at all these hierarchies for a hypothetical GTK+ application exposing information and a screen reader accessing it:

As you can see, there’s an ATK tree matching the GTK+ hierarchy, and another AT-SPI tree matching the ATK one. Finally, the screen reader accesses the information through that AT-SPI tree, as explained above.

Accessibility in WebKitGTK+

Now that we already understand the basics of accessibility in GTK+ applications, let’s add the bits specifically related to WebKitGTK+. Namely:

WebCore’s Accessibility Objects
WebKitGTK+ (ATK) wrappers
WebKit2GTK+ specific details

Again, a picture is usually better than just text, so here you have one too:

In order to clarify it a bit more before explaining each point, let’s just say that you’ll have to look in the dashed box named WebCore accessibility world, where the hierarchy on the left (red & orange) represent the WebCore Accessibility objects, while the one on the right (the green one) represents the WebKitGTK+ ATK wrappers.

With this in mind, let’s examine these three points in more depth:

WebCore Accessibility objects:

Similar to GTK’s Accessibility Implementation, WebCore‘s accessibility objects are the implementation of an independent hierarchy exposing accessibility related information for objects present in a web page. As the mission of accessibility in WebKit is to expose information to users that are normally being rendered in the screen (as well as some other information that might be hidden to regular users), there is a tight relationship between this hierarchy and other ones in WebKit, such as the DOM tree and the Render Objects tree.

This layer is meant to be platform-agnostic, so you won’t find much WebKitGTK+ specific stuff here. Instead, you will find the implementation of the accessibility related specifications published by the W3C‘s Web Accessibility Initiative (WAI), such as WAI-ARIA.

WebKitGTK+ ATK wrappers:

An ATK-based implementation of an accessibility hierarchy where every ATK object will take care of wrapping the proper accessibility object from WebCore, as well as implementing the proper ATK interfaces depending on the situation (e.g. the role of the WebCore accessibility object, some properties coming from the associated Render Object…).

The ATK hierarchy created here is connected with the ATK hierarchy from the embedding application (normally a GTK+ app) by setting the root ATK object in this tree (normally representing DOM‘s root element) as the child of the leaf ATK object in the tree coming from the embedding application (normally the GtkWidget containing the WebView).

As is the case with any other regular GTK+ application, this ATK hierarchy will finally be seen by ATs thanks to the translation that the ATK <-> AT-SPI bridge will do for us, making the whole ATK tree from the WebKitGTK+ based application (from the top level GTK+ window down to the deepest accessibility object inside WebCore) available to the AT-SPI registry by means of D-Bus.

WebKit2GTK+ specific details:

I already talked about this in previous posts, so I will focus here just on commenting the main difference compared to the generic case for WebKitGTK+ described earlier (see previous diagram above):

WebKit2GTK+ implements a split-process model, where the high level API belongs to one process (the UI process) while the core logic of the web engine lives in another one (the Web process).

From an accessibility point of view, this means that the full hierarchy of ATK objects we had before is also split in two parts: some accessibility objects are now in the UI process and the rest of them will be in the Web process.

To be more specific, we’ll find the following objects in each process:

UI Process: basically, the ATK object associated with the GtkWidget provided by WebKitGTK+, that is, the WebView.
Web Process: the whole hierarchy of ATK objects wrapping the accessibility objects from WebCore.

As I explained previously, these two ATK hierarchies will be seen as a single accessibility hierarchy by ATs thanks to the “magic” of AtkPlug and AtkSocket classes, which takes care of exposing everything together in a single AT-SPI tree. And remember that such a tree is modelled by means of D-Bus, so it does not matter that things are actually in different processes.

Thus, since ATs just understand AT-SPI, they will see The Right Thing ™ as in the previous case where we have one single process. See the following diagram for a more visual explanation of this:

Wrapping up

So that’s it. At the end the post turned out to be longer than what I was expecting, as my initial idea was to publish the stuff I wrote internally at Samsung this week, but ended up extending it quite a lot!

At least I hope this will be helpful for anyone willing to contribute to accessibility, either in WebKitGTK+ or in a more general way.

After all, most of the stuff I talked about here applies to every accesible GTK+ application: Assistive Technologies (ATs), AT-SPI, ATK, the ATK <-> AT-SPI bridge…

Last, I would like to thanks Joanmarie Diggs from Igalia for her help with this blog post. One certainly feels more confident writing a long article like this one about a very specific topic when you have one of the most experienced persons on the matter reviewing it!

GUADEC, WebKit and bikes

Published July 20, 2012 / by mario

It seems this year GUADEC is going to be pretty close to my place and so I will surely attend, but this time I won’t go by plane but by bike, which since some months ago has become my main vehicle for moving around the beautiful city where I live in: A Coruña.

Also, besides hanging around the venue and trying to help as much as possible as the local I am, I’ll be talking about WebKitGTK+ in the afternoon on Thursday 26th, so feel free to come round the room if you feel curious about the current status of the whole thing and the current plans for the short and medium term, which are mostly focused around WebKit2 and the roadmap we’re already following.

You probably already read some news related to this coming from my mates in the Igalia WebKit team, (like the improvements in Accelerated Compositing or the migration of our handsome browser Epiphany to using WebKit2), yet I will try to deliver an interesting talk to y’all. I just hope I’ll be able to do it (but please forgive me if I don’t).

So that’s it. As usual, just feel free to talk me if you see me around if you want. I’ll basically be around the venue most of the time during GUADEC, and will attend a11y and WebKitGTK+ BoFs on the 30th and 31st, so I’d say it will be pretty easy to find me.

Back to FOSDEM

Published February 1, 2012 / by mario

So it seems I’m going to FOSDEM this year (yay!), together with a bunch of other Igalians who will be attending as well, coming from different places from across the globe (well, mainly from Europe this time).

I know some people will probably disagree with me on this, but for me FOSDEM is one of the greatest events of this kind, and so I’m quite happy to go there this time, specially after not being able to attend last year due to some unexpected (and unavoidable) personal matters.

Opposite to other occasions, this time I’ll be there not only as an attendant but also as an speaker, talking about WebKitGTK+, its status and the roadmap of the project towards WebKit2 (the split process model “flavour” of WebKit), together with my mate Philippe, on Sunday afternoon. Thus, for the first time ever, nobody will be able to accuse me of going there just because of the beer event, which wouldn’t be true anyway.

For the impatient ones, the talk will be mainly about reporting on the work done during the last months in “WebKitGTK+ land“, as well as on the stuff that is already planned for the upcoming releases. Good examples of those would be, for instance, the ongoing effort to add support for Accelerated Compositing, or just the new features related to WebKit2GTK+ such as, of course, the solution for enabling accessibility support there. Ah! And of course, we’ll try to run some demos there too… fingers crossed!

Besides, I’m of course looking forward to meeting some people I haven’t seen for a while now (haven’t attended to the latest Desktop Summit either, due to very good reasons too), so if you see me around and want to chat and/or meet for a while, just let me know. I must look shy, but it’s usually a matter of minutes (seconds?) for my shyness to go away…

So that’s it. Just a final line to say “thanks” to my company for fully sponsoring this thing.

See you in Brussels!

Accessibility support in WebKit2GTK+

Published January 27, 2012 / by mario / 1 Comment on Accessibility support in WebKit2GTK+

As Piñeiro already mentioned in some posts, last week a bunch of hackers attended the ATK/AT-SPI Hackfest 2012 here at the Igalia offices, in the lovely city of Coruña.

As the guy working on accessibility support for WebKitGTK+, I attended the hackfest to join some other great people representing different projects, such as Mozilla, Orca, AT-SPI, ATK, GTK+ and Qt. So, apart from helping with some “local” organizational details of the hackfest and taking some pictures, I spent some time hacking in WebKitGTK+‘s accessibility code and participating in some discussions.

And from that dedication I managed to achieve some interesting things too, being my favorite ones a big refactoring of the a11y code in WebCore (so it’s now better organized and hence more readable and easy to hack on) and pushing my patch for enabling accessibility support in WebKit2GTK+, after going through a meticulous process of review (see the related WK bug), which started with the patch I wrote and attached back when attending to the WebKitGTK+ hackfest, as I mentioned in my previous entry in this blog.

Yeah, I know that some weeks have already passed since then and so perhaps you’re thinking this could have been done faster… but I’ve spent some weeks on holidays in Barcelona in December (pictures here!) and so I wouldn’t have much time before January to devote to this task. However, the patch got integrated faster than what I would expect when I proposed the first version of it, so I’m quite satisfied and happy anyway just by being able to announce this at this moment. Hope you share my joy :-)

So, what does this mean from the point of view of accessibility in GNOME? Well, that’s an easy question to answer: from now on, every browser that uses WebKit2GTK+ will be as much accessible as those using the previous version of WebKitGTK+, and this is definitely a good thing. Of course, I’m certain there will be bugs in this specific part that will need fixing (as it always happens), but for the time being this achievement means “yet another thing less” preventing us from pushing for upgrading some applications to switch to WebKit2GTK+, such as devhelp (some ongoing work already done, as my mate Carlos announced yesterday), yelp, liferea… and the mighty Epiphany browser, which is rocking more and more ech day that goes by.

Last, I’d like to share with you an screenshot showing this new stuff, but as I am a little bit tired of always using Minibrowser (that small browser we use for testing WebKit2), so I decided to try instead that new branch Carlos recently pushed for devhelp, so you could check that what I mentioned before is actually true.

So here you have it (along with a couple of additions done with Gimp):

As you can see, devhelp is running and Accerciser is showing the full hierarchy of accessible objects associated to the application, starting in the UI process (GTK+ world) and continuing in the Web process, where all the accessible objects from the WebKitGTK+ world are being exposed. As I explained in a previous post, the magic making possible the connection between the two process is done by means of the AtkSocket and the AtkPlug classes, also represented in the screenshot attached above.

So, that’s it.

WebKitGTK+ Hackfest: WK2, a11y and Ephiphany’s ad blocker extension

Published December 6, 2011 / by mario

Some posts have been already published about this during the last days, but just in case you missed them I will mention it here again: Last week, a bunch of hackers gathered together in the Igalia office in Coruña for the third edition of the WebKitGTK+ hackfest , and a lot of work has been done, as Juanjo has already summarized in his “WebKitGTK+ hackfest wrap up” post.

So, as everything has been already said from a more general perspective, I’d like to write my very personal wrap up here, focused on the tasks that I’ve been working on, which can be summarized in three:

Enabling accessibility support in WebKit2GTK+.
Rewrite of the Ad Blocker extension for Epiphany.
Bug fixing in WebKitGTK+‘s accessibility related code.

Enabling accessibility support in WebKit2GTK+

This has been, by far, the task I devoted most of the time to during the hackfest, mainly focused on writing a ‘feature complete’ patch that could be applied upstream, and thus that could be reviewed in first place. But, what do I mean by “a ‘feature complete’ patch”? Well, perhaps you are already aware of the initial results already got in the WebKit2GTK+ a11y realm, but those results were obtained with a patch still in a very early state and, among other things, lacking a very important requirement for getting it accepted upstream: tests.

Fortunately, I can now proudly say that I managed to find a good way to write those tests (specially tricky due to the multiprocess architecture of WebKit2) and that there shouldn’t be any problem either with getting them work properly in the buildbots, which was something I was quite concerned about by the begining of the week, to be honest.

Besides the tests, the other obvious problem was that such a patch was not widely tested yet with the Orca screen reader (I use Accerciser for development purposes most of the time), and that would for sure unveil issues that would need fixing before being really able to propose a patch for reviewing, and so that was the other aspect where I put the spotlight during this week.

And regarding to this, I have to say that Joanmarie Diggs was working tirelessly by testing Orca with my WebKit2GTK+ a11y patch, reporting bugs, and helping me a lot to prioritize the tasks that would need to be done. From all those, I mainly worked this week in the following ones:

Emitting the AtkDocument’s signals (‘load-complete’, ‘load-stopped’ and ‘reload’), which was working only in WebKitGTK+ but not in WebKit2GTK+. See the bug report and the patch (still pending on review) for this issue in bug 73750. Also, I reported and worked for a while in another bug related to this, which is now already fixed upstream (see bug 73746). Yay!
Ensure that the accessibility hierarchy doesn’t break when (re)loading, which was causing that Orca stopped speaking unless it “manually” drilled down the full a11y hierarchy after the (re)load. I finally fixed that issue yesterday and integrated it in the patch for enabling a11y support in WebKit2GTK+, now already attached and pending on review along with bug 72589.

So, the conclusion of this part would be that we have now a patch in WebKit’s bugzilla (see bug 72589) that, once it’s approved, would enable accessibility in WebKit2GTK+ once and for all. Of course, this will probably take some time before it gets accepted upstream, but it’s yet another nice milestone in my opinion, and I personally hope it would happen on time for GNOME 3.4. Time will tell, though.

Rewrite of the Ad Blocker extension for Epiphany

This was another thing I’ve been randomly working on since some time ago (whenever “spare” time permitted), and that I was able to advance quite a lot right after coming back from the parental leave I enjoyed on September (did I say my second child was born on August the 30th?). However, the patch was not finished by any means, and some issues kindly pointed by Xan in bugzilla needed fixing before being able to say aloud something like “hey, the new ad blocker is now in town!”.

Thus, we thought it would be good to devote some time during the hackfest to try to close this task too, so we did: Xan reviewed the new version of the patch (addressing the issues he previously pointed out), I made some last changes based on that new feedback from him and we finally pushed it to the repository, replacing the old ad blocker extension with this new one, which is based in Midori‘s ad blocker and so is compatible with Adblock Plus filters, which work very well IMHO.

So, this basically means that the new ad blocker extension will be present from Epiphany 3.4 on. Check out the related bug in GNOME‘s bugzilla: bug 660154

Bug fixing in WebKitGTK+’s accessibility related code

Besides working in the WebKit2GTK+ a11y realm and on finishing the new ad blocker extension, I’ve also spent some time (although not as much as I would have wanted) fixing regressions in WebKitGTK+‘s a11y code as reported by Joanie (basically bug 72804 and bug 72830).

Compared to the other two points, this has been of course a pretty small contribution, but worth doing anyway since they were very important for Orca to work properly with WebKitGTK+ based browsers (special mention to bug 72830 here).

Conclusion

From the work-related point of view, I’d say this hackfest has been highly productive in general, as we achieved many goals which, as Juanjo pointed out in his wrap up post, “were not mainly about fixing critical and blocker bugs and implementing basic missing features, but about more ambitious and challenging” ones. As for me, I’m pretty happy with the results I got, specially with the WK2 a11y patch, which has now a much better shape, and so I hope we can integrate it soon upstream.

And from a more personal point of view, I’d like to say I had a great time (again!) this year in the hackfest, and not only because of the achiements got, but also because I had quite a lot of fun as well, because I met new people and because I felt, more than ever, part of a community and a project which I love.

To finish, I’d just like to mention that I’ve been taking some pictures during the hackfest, which you can check out in this photo set in flickr (pictures uploaded with Frogr, of course!). Nayan has also taken some pictures as well, check them out here.

Of course, thanks a lot to the sponsors that made this possible: Collabora, Motorola, Igalia and the always awesome GNOME Foundation. I hope we’ll be able to repeat it next year, since this hackfest it’s only getting more and more awesome every time it happens.

Orca and WebKit2GTK+: initial results

Published November 11, 2011 / by mario

Last May, I wrote about some initial tests I did back then with AtkSocket and AtkPlug, just to learn a bit about that API that allows connecting accessibility trees in different processes, thanks to the magic of the ATK bridge. I did that in order to prepare for the work that would probably be needed when adding accessibility support in WebKit2GTK+, so browsers using the new version of this web engine could be at least as much accessible as those using the single-process WebKitGTK+ library. Looking back, I think that the effort has definitely payed off…

However, because of one reason or another (and not necessarily work-related), I was not able to devote much time to keep working on this until some weeks ago, but fortunately I’m now again working on this as my primary task in Igalia, as part of our WebKit team, so I expect to have some nice results soon, hopefully also in the form of integrated patches upstream.

And talking about results, I can’t avoid sharing the following at this very same moment, which is actually the trigger that made me write this post: today I was able to make Orca read, for the very first time, web content rendered through WebKit2GTK’s MiniBrowser, which has been not an easy task since the multiprocess architecture of WebKit2 made it a little bit challenging, to say the least.

Of course there’s still much to do in this regard, so do not think of the current status of the task as it was nearly finished or anything… but still I thought it was a nice milestone to share with the world, and what a better way to do it than using a video for that. After all, Orca‘s main job is about speaking, right?. So here it is:

Orca and WebKit2GTK+: initial results (in Vimeo)

By the way, I’d like to make the most of this post by saying I’m really proud and happy to have Joanmarie Diggs aboard in Igalia for a few weeks now, who contributed a lot to making possible that I was writing this post today. Thanks , Joanie!

WebKit Contributors Meeting, sockets & plugs

Published May 5, 2011 / by mario / 1 Comment on WebKit Contributors Meeting, sockets & plugs

Last week some of the members of the Igalia WebKit team, attended to the second edition of the WebKit Contributors Meeting in Cupertino, California, in order to gather round with other WebKit contributors to discuss and work around our favorite web engine, away from IRC and with a more personal and “human” touch.

As for me, it was the first time I attended this unconference and I have to say it was a great experience overall, even if accessibility (the field I usually work on through the GTK port of WebKit) was not precisely a hot topic there. But in the other hand, I managed to put some faces to people I just knew from IRC, attended to several interesting discussions and sessions, did some actual work ™ and met Chris Fleizach, the guy from Apple involved in accessibility for the Mac port, who attended the meeting on Tuesday morning and discussed with me some interesting topics, mainly about the implementation of the accessibility support in the Mac port of WebKit2.

For those of you that are not up to date with what WebKit2 means and what’s the current state of the whole thing, I’d recommend you to take a look to the WebKit2 wiki, although I can already advance you that one of the main features (if not the main one) of WebKit2 is the new multi-process architecture, which will go bundled right along with the engine, instead of doing the split in the final application, like Google Chrome does now with WebKit1 (see the wiki for more details, and some nice diagrams too, like the following one).

The problem however, from the accessibility point of view, is how to expose such a new architecture to Assistive Technologies (ATs) while, at the same time, having those ATs seeing just one process (the UI process), regardless of the different processes that would be running on each of the tabs (the Web proceses). In other words, how to “hide” to the ATs that they are extracting accessibility related information from a multi-process based browser, so they still keep the illusion of having only one application exposing accessibility objects, as it used to be so far.

Talking to Chris about how they implemented this in the Mac port, he told me they basically needed to make up a way to transparently connect the UI process and the Web Process, so navigation between the root accessibility object in the Web Process and the “leaf” accessibility object in the UI process were done in a seamless way, through a mechanism that would basically allow bidirectional communication in the same way it used to be when there was just one process running. This makes a lot of sense, if you ask me, since you “just” (saying it as if it were not a complex task, even though it actually is) add this mechanism to WebKit2 accessibility code and you’ll automagically get your ATs working as they used to work, without any other changes needed from their side.

However, when it comes to ATK/AT-SPI, which is what we currently use in GNOME to expose accessible objects from applications to ATs, it looks like such a mechanism (or something pretty similar) is already available by means of the AtkSocket and the AtkPlug classes, both subclasses of AtkObject, which basically act as a bridge that allows to connect two AtkObject‘s so the children of one of those (the Plug) are exposed as children of the other one (the Socket), no matter they’re in the same process or in different ones. And this, unless I’m missing something, is exactly what we’d need in WebKit2, probably along with some other things and tweaks that I just can’t think of at this moment. But at least is definitely a very good start point, IMHO.

But… how to communicate those sockets and plugs if they are in different processes? You might be wondering… Well, if you are using the D-Bus implementation of AT-SPI, also known as AT-SPI2, the needed bits for that (the implementation for the atk_socket_embed() and atk_plug_get_id() functions) are already implemented by the ATK bridge (at-spi2-atk), so the only thing you’ll need to do is to provide the remaining implementation of the AtkSocket and AtkPlug classes, register those new AtkObject‘s as the right accessible objects for your widgets (or whatever your “normal” objects are) and to use the ID that at-spi2-atk provides for the AtkPlug object to connect from the AtkSocket object… and you’re done.

However, I have to say that the documentation I’ve found so far about AtkSocket and AtkPlug is not precisely very detailed, so I basically ended up looking directly at the code and trying to write myself a small example to better understand things. And at this point, I asked in the #a11y channel in GIMPNet and it was Mike Gorse who kindly handed me out a nice tarball with an example they had written in C#, which was exactly what I was looking for. Yay! Thanks, Mike!

However, I’d be more interested in a plain C, GObject based, implementation of that example which would serve me both to better understand how it works and to use it as the base for further tests in WebKitGTK, so I went ahead and wrote it, and this is how the results look now, as seen through Accerciser‘s eyes:

As you can see, there are two processes running, ta-socket and ta-plug, and in ta-socket we have an AtkObject named “The Socket” which is exposing, as his only child, the whole subtree present in the other process ta-plug, starting in another AtkObject named “The Plug”… which is exactly, although of course at a much smaller dimension, what I think we would like to see happening in the accessibility tree of any WebKitGTK based application in the future, once that WebKit2 is mature enough to replace WebKit1.

Obviously, this shouldn’t be taken as a “we’re done with WebKitGTK/WebKit2 when it comes to a11y” comment or the like. Actually this is just the beginning of the whole thing… But I think, or at least I would like to think, that is at least a good start point :-).

And before you shoot… “yes”, I’m already planning to help improve the ATK documentation in this regard with the knowledge I acquired while working on this examples, even perhaps next week during the ATK/AT-SPI hackfest (it’s actually one of the proposed tasks) I’ll be attending here in Coruña, at the Igalia offices.

By the way, feel free to grab the source for this example from its git respository at gitorious:

git clone http://git.gitorious.org/atksocket-atkplug-example/mainline.git

Some updates on frogr 0.4 and myself

Published January 22, 2011 / by mario / 3 Comments on Some updates on frogr 0.4 and myself

Too many things have happened to me during the last weeks (some good and some bad, as usual), but I still have found some time to continue improving frogr towards the 0.4 release, which I hope it’s gonna be, at least thinking of my very particular use cases, a very complete and functional release. Still, the UI won’t be great (I’d really need help with this), I know that… but I said “complete and functional”, right? I do not remember having mentioned “beautiful”, “eye candy”, or the like… that’s a matter, though, I’ll probably consider for following releases, but not for now.

The point of this post is that I declared yesterday the official start of the feature & string freeze phase for frogr, which basically means that what you can find in the NEWS file is exactly what you’ll find in the next release (which should happen in 2-3 weeks time), and that the awesome people in the GNOME translations teams can now work on adding new translations, or just updating the ones already present in the Damnes Lies platform, without having to worry about the strings changing again before the release.

Another thing I’d like to publicly say here and now, and which you might have already noticed from the previous paragraph, is that this release is gonna be the first one after having moved frogr to the GNOME infrastructure, which basically means that its previous site and mailing lists at Google Code, as well as the old repository at gitorious, are now deprecated in favour of live.gnome.org, GNOME’s bugzilla and GNOME’s git repository. And this, at least in my very personal opinion, is really great news for frogr, and I’m really happy about it.

Other than that, but related, last week I’ve finally decided to apply for the GNOME Foundation, and got accepted, which was also great news for me, since it was something already in the back of my mind for some time, and I guess I just needed a ‘trigger’ for daring to apply for it… and moving frogr to GNOME, together with all the a11y related work I’ve been doing during the past months in WebKitGTK+, as part of the WebKit team at Igalia, were actually good ‘triggers’ for that.

So a big ‘thank you’ to all who helped making all that possible, specially to Claudio Saavedra for encouraging for doing this moves, and to Christer Edwards, who kindly attended all my continuous requests to perform all the related tasks in record time.

By the way, as Claudio and some other workmates, I’ll be attending to FOSDEM once again this year thanks to the support from Igalia, where I’m starting to have a hard time sometimes to explain that the Beer Event has nothing to do with me willing to go every year :-).

Anyway, apart from hanging around the venue, I’ll also be giving a talk about a11y in WebKitGTK+ in the accessibility devroom on Sunday, so that makes another pretty good reason for me to go this year. So, you see? It’s not only about beer!

Update [05/02/2011]: At the end, and due to unexpected and very important personal matters, I was finally not able to attend FOSDEM. Hopefully I’ll be able to go next year, but as for now I’d like to wish all the attendants of the 2011 edition have a great time there and enjoy this awesome meeting, which is one of my favorite ones every year.

WebKitGTK+ hackfest 2010

Published December 13, 2010 / by mario / 8 Comments on WebKitGTK+ hackfest 2010

After the daily reports written by Diego in his blog, few more things can be told about the WebKitGTK+ hackfest hosted at the Igalia offices last week, but I’d like to comment anyway some impressions from my personal point of view, if you don’t mind reading them.

First of all, this was the second time I attended to this hackfest (I “kind of” attended last year hackfest as well) but now things were pretty different for me, basically because one year ago I was not part of the Igalia WebKit team yet, hence my contributions in the hackfest were pretty small (see my post back then for more details). However, this time I attended full-time to the event and I must say I’m really proud of the work I’ve been doing right there, which I hope will eventually lead to the resolution of this WebKit metabug, which was about fixing bugs blocking ORCA support from WebKitGTK based applications.

But fortunately, the work I’ve been doing during the last week was just a pretty small and humble contribution compared to all the work that has been done by the rest of the people attending to the hackfest, like fixing GTK3 and GObject Introspection issues, fully integrating in libsoup all the new cache stuff written for WebKitGTK (which eventually lead to removing the equivalent code from WebKitGTK, as my mate Sergio told some weeks ago, as soon as some bots upgrade to the latest version of libsoup), adding support for profiling in JavascriptCore, implementing some missing and advanced features into the DumRenderTree (aka DRT, the so beloved tool for writing functional tests), fixing spell-checking support… and bugfixing in general (as well as, most likely, lots of other things I’m failing to recall right now). You can read Diego’s blog for more details on those.

Other than that, there was also time for working in Epiphany were some notorious fixes and improvements also happened. Those I can remember right now are the new error pages for epiphany, the implementation of a certificates viewer and new font preferences, getting rid of GConf in epiphany-extensions and general bugfixing tasks. As you can easily understand, as the devoted and committed Epiphany user that I am, I’m pretty excited with these improvements as well. Not needed to say anything about this patch committed at the beginning of the hackfest, I guess, in my opinion this is one of those cases where a picture is clearly worth a thousand words:-)

So, as you can see it was a quite productive week after all here in Coruña!

Last but not least, I’d like to specially thank The GNOME Foundation for sponsoring the event, as well as Igalia and Collabora for helping make this possible once again. Hope we can repeat it next year, and that more people will join the event to help making WebKitGTK an even better web engine for the GNOME platform.

See pictures of the hackfest here:

Wrapping up the GNOME a11y hackfest

Published October 13, 2010 / by mario / 1 Comment on Wrapping up the GNOME a11y hackfest

As I told in my previous post, I’ve spent the last week in Seville attending the GNOME Accessibility Hackfest, which was an amazing experience to me, so I guess now it’s time to talk about some (mostly personal and subjective) conclusions.

First of all, let’s repeat this was the first time I attended an a11y related hackfest, and from that point of view the experience was even better and more rewarding than what I’d expected. Meeting other a11y developers in real life and putting faces to them, along with being able to check the real needs that people demand from assistive technologies was an awesome experience, and way richer than just having a list of bugs in need of getting fixed.

Up to this point, I can say that one of the best things I extracted from the hackfest is that I’ve learned, with real-life living examples, why implementing certain kind of stuff in the WebKitGTK port is so important for people with diverse kind of impairements, and even learned to prioritize and value different a11y related issues so I could make the most of my work to be more effective when helping, with my humble 2 cents in my daily work, to make the next release of GNOME a good one also from the point of view of accessibility. Obviously, there’s still a long road ahead to keep learning and improving, but I think this is a good achievement anyway, if you ask me.

Other than that, together with Joanmarie Diggs, we managed to move some important things forward related to the integration of WebKitGTK based applications and the ORCA screen reader, so now the “ORCA experience” with the web is clearly better compared to how it was some months ago (or at least that’s what Joanmarie says, and I trust her). In this regard, I feel specially proud of the advancements we’ve been doing with the implementation of the AtkHypertext/AtkHyperlink related stuff, which makes the experience of “screen reading” a web page way better and more complete than how it would be without that feature. And I mention this stuff here, because that’s precisely what I’ve been working on most of the time during the hackfest and because, even though it’s still a provisional patch pending on some final tweaks to be proposed for review, I hope it will be soon integrated in WebKitGTK…. (you know, ‘hope’ is a so beautiful word… but I feel somehow confident with this stuff, “let’s hope I’m right” :-)).

So, as you can see I have reasons to be very happy about the results of the hackfest, even if we all were somehow jinxed because one reason or another, and the overall result of all this stuff is that I clearly felt a boost in my motivation to keep working on this stuff and to keep helping to improve the overall status of the accessibility from my beloved WebKit world… which is not that bad, IMHO.

Last but not least, I have to say there was room as well for having a lot of fun (“All work and no play makes Jack a dull boy”) with my mates from the GNOME accessibility team, who even had prepared a surprise for me in the shape of a cake to celebrate my birthday (see the picture below, by Juanje Ojeda).

Thank you guys! It was a really really nice experience! Let’s repeat it soon in the future… but as for now, let’s get back to work!

Attending to the GNOME a11y hackfest

Published September 27, 2010 / by mario / 2 Comments on Attending to the GNOME a11y hackfest

As my mate Alejandro said back in June, next week the 1st AEGIS international conference will take place in Seville, Spain, and there I’ll be attending, together with him, the GNOME Accessibility Hackfest that is planned to happen there as a parallel event to the conference (see AEGIS web page or GNOME live wiki, for more details).

As for me, this is the first time I attend to an a11y-related hackfest and I must said I’m quite excited about it, as been there should be the perfect scenario to work hard on moving things forward related to a11y support in the WebKitGTK platform, which has been what I’ve been working on most of the time during the last months here in Igalia, as part of our WebKit team.

On top of that, this is also going for me the first time I’ll meet in real life some people I usually work with in a regular basis since I started working on a11y-related stuff in WebKitGTK, such as the always helpful Joanmarie Diggs, who patiently answers all my doubts about those bugs blocking ORCA support, for instance.

Because of all this, I have great expectations about this hackfest, both as a personal experience (from the most selfish point of view you can imagine) and as a great boost to all the work being currently done to improve GNOME as an accessible platform towards the GNOME 3.0 release.

Can’t wait!