In the coming days, I’ll try to write a longer post with more technical details about my recent work on the project. Until then, here’s a short screencast showing some of the new features; interactive table resizing and cell selections. This time it includes me speaking a bit (!). And the format is now WebM/VP8 instead of Ogg/Theora. White areas in the capture got some weird tinting, but that’s a screencasting problem and not my code :)

Download Video (~12 MB WebM)

Bye ’til next time!

Just a quick follow-up post to give a happy end to my previous woes. Without further ado, I give you Scribus rendering all 43 cases of border joins mentioned in the previous post (I had forgot two of them before):

Cheers.

Some Background

Since a table cell may span multiple rows or columns, along each side of the cell, it may share its border on that side with N neighboring cells, or with the table border. Along each such shared border segment, the neighboring border along that segment must be identified and collapsed with the cell border according to certain rules. The most commonly used rules are those specified in the CSS collapsing border model, sometimes with slight modifications. This is also what I’m aiming for in my implementation.

Lines in Scribus can traditionally be represented by an arbitrary number of lines, each with its own color, width and style, drawn on top of each other, thin over thick. Like this:

A multi-line in Scribus

In trying to keep in style, I’d of course like to support these types of lines in my implementation of table borders. This is also supported by competing products such as InDesign.

Borders from different cells, or from the table itself, meeting at an intersection in the table should optionally be joined. Joining is the process of adjusting the start and end points of the border, as well as adjusting the start and end points of the individual lines constituting the border, in order to make a “best effort” join with any other borders meeting at the intersection.

This is where the fun begins. I’ve identified at least these 41 possible cases of joins:

Table Border Join Cases

The Past ~Two Weeks

In the past two weeks most of my work has been trying to find a joining/painting algorithm that correctly identifies all the cases above and performs the necessary adjustments.

To paint an entire table, the painting algorithm must iterate over all cell edges in the table, and for each edge, iterate over all shared border segments. For each segment, the segment is collapsed with the correct neighboring border. Next, each of the, possibly six, other border segments meeting the segment at its start and end point must also be identified. This means identifying all the cells surrounding the segment and collapsing the appropriate shared border segments between them.

Let’s take a simple case as an example. In the example below we want to paint the top border of the green-tinted cell, which spans two columns. The thin red dotted line represents the underlying table grid.

Painting a Top Border

In the first iteration above, in addition to collapsing the shared border segment between the cell itself and the cell above it, the five border segments coming in to meet it at the two intersections must be identified and collapsed correctly. After that, adjustments for joining can be made to the segment start and end, before the segment is finally painted.

Similarly, in the second iteration, there are four additional collapses that needs to be done before joining adjustments and finally painting can be done.

Needless to say, it’s been quite a chore trying to get this to work. Especially the joining algorithm has been a tough nut to crack. I’ve used up numerous sketch pads trying to figure it out. When working on something like this, pen and paper is invaluable. But, although there are some cases it can’t quite handle in a pleasing way, I think I finally have an approach that will work. I’ve intentionally made the code for collapsing and joining strictly separated from the rest of the code, to ease unit testing.

To not get too complicated the algorithm I’ve settled on imposes a strict painting order — horizontal borders must be painted on top of vertical ones. This means two iteration across the table. Iteration is quite fast though, and besides, I’d rather spend my time optimizing cell accesses on the table than convoluting the joining algorithm with added complexity.

So without further ado, here’s a screenshot of some collapsed joined and non-joined borders on a table in Scribus:

Joined Borders in Scribus

Although there are some bugs in there, I have other fish to fry at the moment, so I’m going to leave painting for a while. And if there’s anyone out there who, after looking at that picture with the 41 join cases I’ve identified, get a brilliant idea for an algorithm that covers them all with a minimal amount of code, then contact me! Please!

That’s all for now. Bye ’til next time!

• Work Report, a report on what I’ve been up to since my last report.
• Project Status, how I’m standing with regard to the schedule in my project proposal.
• Upcoming Work, a rough outline of my next steps.

As this is the very first blog post about my project, I’ll start with a little introduction with general information about my project.

Introduction

Tables provide a compact way of representing tabular information in a page layout, and is a common design element in books, magazines and advertisements. Scribus currently has a shortcut for creating an automatically grouped table-like array of text frames. These are not proper tables. To stay competitive, Scribus needs better support for tables. Support that enables the user to create and edit tables as distinct items with their own set of properties as commonly found in other table implementations. This is the rationale for my Summer of Code project, in which I’ll add a new table page item to Scribus.

First glimpse of new tables

The user should be able to perform common table operations such as inserting rows and columns, merging and splitting cells, formatting borders and backgrounds as well as editing the text of the table. The table will represent the text of the cells using standard Scribus text frames.

More information can be found in my project proposal.

So without further ado, here follows my first report.

Work Report

• Bootstrapping: Drafted an initial design document including some mockups on the Scribus wiki and discussed the ideas in detail on the scribus-dev mailing list. Also set up a development environment (using Qt Creator this time even though I’m a die-hard Vim junkie).
• Added a skeleton of a table style. This was way back in April, just to see what work is involved in creating a new type of style. The style is the most basic one imaginable. At the moment I think it has a single property for background color. Instances of the new table style can be managed in the Style Manager. It doesn’t do style inheritance yet, though that should be relatively easy to fix.
• Added class PageItem_Table, a new page item for tables. The class represents a grid of table cells by keeping a few lists with row / column geometries.
• Added ability to insert a new PageItem_Table from toolbar or Insert menu followed by mouse dragging. Just like the current “tables”.
• Added API to PageItem_Table for the basic table operations
  • insert and remove rows and columns,
  • resize rows and column.
• Added support for merging cells to PageItem_Table as well as an accompanying helper class CellArea used when keeping track of areas of merged cells. The CellArea class also comes with what I think is the very first unit tests in Scribus. The tests are in tests/ and can be runned using make test.
• Added some rudimentary painting to PageItem_Table, just to be able to see the cells. Still a long way to go for proper table / cell painting, which is kind of non-trivial.
• As scriptability allows me to experiment with tables before any UI is done, I’ve added the following new scripting methods. These have been added to the old scripter, but should be easy enough to port to ScripterNG later on.
  • createTable(x, y, width, height, numRows, numColumns, ["name"])
  • getTableRows(["name"])
  • getTableColumns(["name"])
  • insertTableRows(index, numRows, ["name"])
  • removeTableRows(index, numRows, ["name"])
  • getTableRowHeight(row, ["name"])
  • setTableRowHeight(row, height, ["name"])
  • insertTableColumns(index, numColumns, ["name"])
  • removeTableColumns(index, numColumns, ["name"])
  • getTableColumnWidth(column, ["name"])
  • setTableColumnWidth(column, width, ["name"])
  • mergeTableCells(row, column, numRows, numColumns, ["name"])

Project Status

The goal in the schedule for this first week was simply Data structures for tables. Intentionally a quite moderate goal, as I didn’t know how long it would take to get up to speed. So unsurprisingly, I think I’m a little ahead of schedule. Worth noting though is that the tables are currently only skeletal in their nature; they hold no content and are simply a grid of empty cells. This is all according to plan though, and integration with text frames is scheduled for later.

Upcoming Work

The goals for next week according to the schedule is

• Basic table layout with fixed column widths and mock content in cells.
• Insertion/removal of rows/columns.
• Basic drawing of table.

but as at least some of this has already been done, I think I’ll start looking at other things as well. Right now I’m looking at proper table / cell border painting, as well as fixing a couple of bugs in the code I have so far.

I have an installation of Adobe InDesign CS5.5 running in VirtualBox and I’ve been looking at what border model they’re using and how they’re doing border conflict resolution. It seems to be a variant of the CSS2 collapsing border model, which is probably what I’ll try to go for initially in Scribus as well.

That’s all for now folks! I’ll conclude with a little screencast showing me playing around with a table using the new scripting methods. I highly recommend you download the source OGV instead of watching it here, as the quality turned out quite bad in the blip.tv conversion.

I’ll probably blog my progress here. What do you guys think; OK for me to have it aggregated on p.k.o? Scribus KDE-ish enough for you?

PS. For those of you who didn’t understand or can’t be bothered to click the links; I’ve been accepted into GSoC this year to improve the tables support of Scribus. You might remember me doing the same for KWord as part of GSoC back in 2009. DS.

Some of the things that has happened in KOffice tables land since last time is:

• Fixed a bug in relative table width loading.
• Further improved KoTextDebug to work with the new styles for tables, rows, columns and cells.
• Added support for column width and relative column width. Up until now all columns were hard coded to equal widths.
• A KoTableAndRowStyleManager has been added for managing column and row styles.
• Fixed a bug where two subsequent tables were painted on top of each other.
• Added support for minimum row height.
• I began to re-factor the table layout to use a list of rectangles that describes the table instead of a simple rectangle, and started thinking about table breaking. Pretty soon though I realized that I was probably in over my head, and passed the torch to Casper, who has been hard at work trying to get the first pieces in the breaking puzzle, hard row breaks, to work. I just got word on IRC that he may just have it working (!).
• Finally fixed the issue with an empty line above tables. This text block is there by necessity; it’s mandated by Qt Scribe. Me, Thomas and Casper discussed a bit on how to best handle this. And at least for now, we’ve chosen to solve it like Qt does in its own layout engine, which is putting the empty block to the left of the table. We might do some improvements to the user interaction here though.
• Added three new classes; KoTableFormat and its subclasses KoTableColumnFormat and KoTableRowFormat. These are very much like the QTextFormat classes in Qt in that they’re simple implicitly shared classes that hold a set of properties. They are not used yet, but will be soon.
• Fixed bug in justify aligned tables when an explicit width is also specified.
• Fixed up the table layout code to handle nested tables, as it was totally oblivious to them up until now.

In addition to these things there were some other small fixes, and I’ve also worked a bit on the tests, which all needed fixing after Casper’s re-factor. More work to be done on testing though and right now I’m looking at adding some loading tests to the test suite that Girish created.

On Tuesday last week I fell terribly ill, with high fever, and was mostly in bed or on the couch for a good two days. I’m back to full health now though. I think it might have been food poisoning.

The future of the column and row style manager I mentioned above is unsure though. At the moment it’s just a container for pointers to the styles, and the loading code tosses them in there and decorates the QTextTable with a property containing a pointer to the manager. Not really the prettiest of setups, and we need to find a better solution. The format classes I added are a part of this puzzle, but the question is; where do we store these formats?

I’m sure we’ll work it out though, and now I need to get back to coding. Sorry if this post was a little duller than my previous ones, but to cheer you up, here’s a screencast in which I save a set of nested tables in OpenOffice.org Writer and open them in KWord to edit them a bit :)

Nested Tables from OO.o to KWord

Lots of stuff has happened since my last blog. It’s Monday afternoon when I’m writing this and I’m on the train back from the sprint that Casper Boemann and I had at his place outside Copenhagen this weekend.

Since Casper is now working on tables support as part of his employment with KO GmbH, we both agreed that it would be a good idea to have a meeting to make plans for our cooperation during the rest of my GSoC, but also do get some hacking done.

The weekend has been intense to say the least, and we’ve added support for several essential features of tables, such as:

• 
  

  Row and Column Spanning

  Cells can span multiple rows or columns. The row height calculations have been updated to handle the case where a cell that spans multiple rows grows vertically from a text insertion; the last row that is spanned into will grow in height along with the spanned cell.

• Table and Cell Styles

  Thanks to hard work from Casper, we now have KoTableStyle and KoTableCellStyle, which are proper styles for tables and cells, respectively. Work is underway for row and column styles.

• 
  

  Loading

  Casper also took the plunge on loading and quite quickly got stuff up and working, and with the commit he did right before I left his apartment, most of the features here should now be loading correctly from ODF.

• 
  

  Table Alignment and Margins

  Tables should now support all the alignment options in ODF; right, left, center and margins (justified), and they can have a left, right, top and bottom margin.

• 
  

  Table Background

  Tables can have a global background color set.

• 
  

  Relative Table Width

  Tables can now have a percentual width that is relative to the page width.

• 
  

  Cell Background

  Individual cells can have a background color set.

• 
  

  Cell Borders

  Cells can have borders, including double borders with a spacing between them.

• 
  

  Cell Padding

  Individual cells can have a left, right, top and bottom padding.

During our sprint, Casper focused on loading, styles and drawing, and also sat with me for some pair programming when we implemented spanning, while I worked mostly on the layout part and unit testing of the features.

All in all it’s been a great weekend. Of course there are bugs to be worked out, but the basics of these features are all there and working. Casper is a very friendly host and a great programmer. I have a lot to learn :) He also did all the cooking and made me feel right at home. Thanks Casper!

The weekend ended with a couple of hours of planning, and we made a detailed list of ODF table features that we want for 2.1.

It’s quite pleasing to see that the approach to tables in KOffice that we’ve chosen seems to be holding up, and I’m pretty sure it will work out for all the remaining features that we need to implement, of which proper table breaking across pages is perhaps the most tricky one, from a layout point of view. But that’s for later; right now I’m going home for some resting up, and I think Casper is doing the same back at his place.

Anyway, kids are screaming like crazy here on the train, so I think I’ll end this blog with a little screencast showing off some of the new features, and then put on my headphones and doze off.

So long!

Copenhagen Sprint Features Screencast

Direct link (2:40 min, 15.2 MB Ogg/Theora)

Right now I’m on the train down to Copenhagen for a little sprint/coordination meeting with Casper. As it turns out, he’ll be working on tables too and therefore we thought it would be good to get together and coordinate a little. And code like crazy of course.

I’ll be going home Monday afternoon, and hopefully there will be something blog worthy coming out of this. *yAwn*. Got up at 5 AM, so a little tired, but no rest for the wicked right? :)

Later, Elvis.

As you may remember from my last post, I was cheating in my screencast; the table height was hard-coded and the cell heights were not adjusted to content height. What I needed to do was to handle the following case:

Cell Heights Adjustment

I have just commited the beginnings of this in r997173. TableLayout now has an updateCellContentHeight(QTextTableCell&) API, which is called once the content of a cell has been laid out in Layout::nextParag().

There are a lot of rough edges, and my tests of course broke left and right, but it principally works, fingers crossed ;) Here’s a video demonstration:

Cell Heights Calculations

Direct link (1:30 min, 7.6 MB Ogg/Theora).

I skipped uploading to Vimeo this time, as it’s so clunky. I don’t think anyone will miss Flash right?

Anyway, my dear mom just appeared and is taking me out to eat, then I need to start fixing/extending those tests, so bye for now!

I hope everyone had a nice time on Gran Canaria. It seems like it’s been a really successful event from all the posts I’ve read.

I’m not one of the frequent GSoC bloggers here on p.k.o, but as we are in mid-term now I thought I’d make a little summary of my thoughts around the project so far and give a little status report.

Initially my idea was to divide this post into two sections; “The Good” and “The Bad”, with a summary at the end. But then I realized that for almost every bad thing I could think of, I could find a good side to it, and vice versa. There’s always two sides to a coin. So instead I’ve decided to make a table. How appropriate ;)

All in all, I’m having a great GSoC so far, and I think it can only get better.

Progress

Now on to some information about my progress. What I’ve been working on is getting some first basic table layout working. The design is quite simple so far, and looks very roughly something like this:

Table Layout Design Overview

The Layout class is the innermost class of the text layout engine in KOffice, and is the one responsible for most of the nitty gritty details of text layout. It is declared as an inner class of KoTextDocumentLayout, which has an instance of this class called m_state.

KoTextDocumentLayout in turn is a higher level class that derives from Qt’s abstract document layout class QAbstractTextDocumentLayout. Qt’s own document layout engine, QTextDocumentLayout, which is not shown in the diagram above, derives from this same ABC. I’ve also included KWTextDocumentLayout, which is a specialized KoTextDocumentLayout used in KWord. It adds support for letting text flow around other shapes, among other things.

What this all means is that KOffice’s layout engine, while still working with an underlying QTextDocument, is completely separate from Qt’s. Now, because of poor research on my part in the weeks leading up to the GSoC start, I actually thought I’d be able to leverage at least some of the built-in support for table layout in Qt. But I have only myself to blame for that, and it’s nothing to cry over now. What I do get is QTextTable, which can be considered a blueprint for a table, and my task is to lay out this table according to its properties, which I’ll later have to extend to support ODF features that don’t have a 1:1 equivalent in QTextTable and its associate classes.

I’ve modified Layout to have an instance of the new class TableLayout, which is responsible for doing layout and drawing of tables. It takes a QTextTable and lays it out, storing the layout information, such as table and cell geometries in an associated TableData instance. It actually caches TableData instances per-table internally. TableLayout offers an API to be used from Layout to obtain cell content rectangles to direct the layout engine into, as well as an API for adjusting cell heights based on the cell contents.

It’s a simple design, but one that I think will work out fine for all the requirements that will come later, and I already have ideas on how e.g. cell spanning should be implemented. Things on my TODO are too many to even enumerate, but what I’m working on right this minute is row heights calculations and finding a good way to let the layout process in Layout notify the TableLayout of cell content height changes. After that comes, among other things:

• Margins/paddings/borders.
• Variable column widths.
• Tests, tests and more tests.

Now let’s see, this post has had a table, a diagram and a list. What’s missing? A video of course :) Take 50 seconds and marvel at me pasting a basic table from Firefox into KWord and editing the cell contents:

Direct link (5.7 MB Ogg/Theora).

Actually I’m cheating in the above video, as the table height is hard coded, but don’t tell anyone! ;)

Finally, I’d like to be honest and admit that progress really haven’t been as fast as I thought it would be, but I do feel that I’m on the right track. And even though the road ahead is longer than what I would have wanted, at least it seems pretty straight, all things considered.

Now if you’ll excuse me, I really need to get back to coding!

Bye ’til next time!