Diversion

So whilst there’s been substantial progress on RD across all tiers (currently doing data architecture and working on PostgreSQL), a problem that keeps cropping up across all enterprises…cropped up again. For me this is a smaller piece of design and development work, which has benefits for a wide range of user groups – and is perfect for the open source model.

Almost feels like a distraction that I’d meant to do something about a few years ago.

Often where organisations want to take the next stage in maturity of their architecture practise, they need look at how they manage their overall enterprise continuum. The starting point is a large volume of flat Visio drawings, a tome of Word documents and probably a whole chunk of Powerpoint presentations. I’d say Writer and Draw but am unsure of how many people would get the reference 🙂

If you’re fortunate enough you may be operating a “living document” approach in platforms like Confluence, but even with some of the diagrammatic markdown there inevitably most of the drawings will still be in Visio. Living document platforms allow you to ditch disperate single document files in order to design & deliver change wiki-style. With the linkage to JIRA it really comes alive of course.

Even really powerful Visio drawings are just drawings. It’s not like there’s a data dictionary or much meta-data behind the shapes – unlike DWG, iServer store or Sparx EA repositories.

What do you have against Visio then?

Nothing at all. It’s a great tool – I remember working for a small architecture & surveying practise in the late 90’s, building CAD tools for them to use in AutoCAD and Microstation. AutoDesk were trying to get their developer network (the ADN) interested in developing on their flat drawing product, Atrix Technical. Intended to be a template & stencil-based CAD tool (sound familiar?), focusing on simplicity vs. AutoCAD-level drawing dictionaries, it was struggling to gain traction with the ADN.

It looked like the biggest problem was that, in an industry already using AutoCAD it was difficult to sell the benefits of a simplified tool. Surveyors just weren’t that interested. At the company I worked for at the time we hit upon the idea of using it for estate agents – when they’re assessing a property it might be a good way of quickly drawing a layout. We import a survey DWG into Atrix as the home layout, then give the estate agents a bunch of to-scale stencils with household items e.g. furniture and appliances.

Sounded good enough that we approached AutoDesk to see if we could get some help with funding for the project – they even came to Nottingham to see us.

For me, that’s where it got weird. AutoDesk have a history of stealing ideas from their developer network and building them into their own product base. It used to be a widely acknowledged practise to ensure you hid all the “good stuff” if AutoDesk were in town. I remember one firm in the early 2000’s built a paid-for standards checker to verify layouts against RIBA and other standards. Saved a lot of people a lot of time and stress. AutoDesk found out and in the next product version it included a standards checker too, killing off the paid-for add-on.

Anyway, back in 1999 I might have expected them to get us to build a bunch of stencils, but build new tools into their own UX. But the senior sales people visiting us were literally only interested in selling us a more expensive developer license with enhanced technical support. Not helping to fund our tiny company building them a new capability in their product.

When they tried to justify it, they claimed that there was no interest in the product, which seemed counterproductive to their own interests. One of their team even let slip that the only use they’d currently found for the “flexible product”, was one of their partners building a stencil set for their Scalextric pieces, then designing circuits for the loft space.

Personally, I thought that was ingenious at the time, but it highlighted the lack of vision AutoDesk seemed to have with Atrix. Then Shapeware / the Visio Company decided that they would offer CAD drawing support for popular DWG and DXF shapes, alongside stencils for software development and UML… then were ‘adopted’ by Microsoft. AutoDesk seemingly gave up.

Despite all of this, Visio was never anything more than a drawing tool. It didn’t get integrated into the design pipeline as a CASE tool, and away from DWG it held no intelligence (material data, usage data) or data dictionaries. It is still an awesome product for quick diagrams. I still use it with the TOGAF Archimate stencils and the Orbus TOGAF stencils for rapid diagrams in front of customers. But it won’t help you create real linkage between your reference architectures and your solution designs, nor will it allow you analyse realisations between your business and application architectures.

The Problem

In 2007 it was relatively easy to get your flat Visio diagrams into Sparx EA (SEA) using either the Sparx MDG to pull, or some VBA and the built-in UML Background Add-on in Visio to export to XMI. Sparx sit on the OMG XMI committee, and SEA users are used to importing and merging XMI docs from their .EAP files into central repositories.

Although you’d often expect to need to remodel a fair amount of what landed in SEA due to the mismatches.

When Visio 2010 arrived this was still just about possible – despite SEA getting into a mess with different versions of XMI. All fine.

Then Microsoft silently killed the Backgound UML Add-on in Visio 2013, whilst Sparx steadfastly failed to keep their MDG up-to-date. By the time the VSDX file format came along, things were getting messy. You would have to export your VSDX as a VSD to use it with the Sparx MDG, and even then there was no guarantee that it would correctly translate your UML or Archimate into a package containing the proper entities in your SEA model. At a client more recently I discovered that they’d been using a standard set of very old UML stencils dating back to UML 2.2. the MDG simply choked with numerous errors when I tried the import.

It seems more able to deal with BPMN but I wouldn’t say that I trust it.

I’ve seen this at literally every client I’ve worked with over the last ten years that have wanted to move to SEA. Visual Paradigm offers some of the kinds of capability I was looking for in Sparx out-of-the-box, although other tools seem to restrict themselves to BPMN imports – or not at all in the case of Orbus’ iServer. All good platforms despite of this.

Sparx still build SEA in 32-bit mode and haven’t built the interface to work with touchscreen well at all – just try using it with a Surface Pen. Imagine not having a right mouse button for a start. Even if they updated their MDG now we’re likely due a new version of Office (and therefore Visio) this year.

The Solution

The best approach in my opinion is to leverage the powerful talents of architects and developers supporting the open source community across the world. For that reason localisation must be baked into the user experience.

Initially the scope must be limited to the direct problems I can see in front of me – transferring Visio UML (using custom stencil sets of an old UML version) safely into a designated package in SEA. Ideally that conveyance will automatically detect existing entities in SEA and add new relationships to elements new to the target SEA repository.

I will undoubtedly want to expand this to BPMN and Archimate in the near future so the mapping mechanism must be customisable by the end user.

Rather than try and extract and transform the XML from within VSDX (which is just a ZIP file containing XML & XSD docs), the more manageable approach is to work within the walls of Visio itself. That means that if any future patches or versions change the VSDX format we won’t have to first notice the problem, diagnose what has changed and then build an adaptation.

We can simply develop using the Visio API via a Visio addon – using the Microsoft supported and maintained VSTO approach. It may mean that the source code needs to be built against specific versions in future, but that’s drastically less work than mapping proprietary VSDX documents. It does create a reliance on Visio licensing but considering the target audience there is a very strong likelihood that all the people wanting to contribute will be both Viso and Sparx users to begin with. Visual Studio Code and Community editions both support VSTO development, although it is restricted to Windows.

Using XMI as the transport instead of SEA APIs will also be easier in the first iteration. Later iterations could revisit this if enough people contribute with that experience (if SEA even provides APIs). I don’t think it’s a good idea to integrate directly with the SEA repository in an RDBMS such as SQLS or PostgreSQL, as we may end up with more problems if the schema changes in future SEA versions – perhaps they’re already different.

If I built an XMI transform assembly I suspect that will be quite useful to many applications, so that should be built as a .NET Core assembly – and therefore x-platform.

The simplest approach with the most vendor support is therefore a Visio addon that exports the current diagram to XMI, using a x-platform library to do the grunt work on the transform.

GitLab

I’ve started work on the design and build already, and have decided that GitLab will provide everything needed in this case. I’ve already created a skeleton wiki there and am adding detail as I get to it – including what is out-of-scope, how it might become in-scope and why.

I’ve decided that the AGPL v3 license looks best to suit this particular toolset.

Whilst I build the basic UX and design the XMI transform mapping, I’ll keep the repository private – if you’re really keen to have a look please express an interest in the comments below (don’t put personal info in there please), and I’ll get in touch via reply. At the moment it’s a one-person job whilst I get all the designs out of my head and onto ~~paper~~ the kanban.

By the time you read this, I may have already made the repository public, and you can take a look here: https://gitlab.com/scribbles-and-dust/visioxmiexporter

Is this something that you or your business would be interested in? Let me know in the comments section below.