S-63 Confusion

[bdbcat]

redcane...

Your understanding is substantially correct. SENC files are manufacturer/device dependent. There is no industry-wide standard for SENC files, so each ECS/ECDIS manufacturer invents their own. OpenCPN has done the same. Our SENC file format is compatible with no-one else.

The reason for the need for SENC files is because S57 is known as a "Transfer format", optimized for cross-platform portability. It is completely suboptimal for rendering of charts on-screen. The design of SENC files has many tradeoffs, especially in the speed/size dimension. OpenCPN elects to make (sometimes very) large SENC files in order to reduce ingestion and rendering time. Incidentally, our SENCs are not machine dependent. They are portable to other OpenCPN instances, machines, and architectures.

IIUC, we would need to encrypt our SENC files on disk to satisfy S63 admins. Otherwise, a programmer of moderate skill could read OpenCPN source code and build a tool to convert our SENCs to any other format desired, a situation which is definitely not desired by the cell publisher.

There may be other places in OCPN where we would need to pass a key/lock challenge as well. S-63 admins will not be happy with open source model, and will need convincing that the proprietary cell information cannot be extracted by a moderately skilled programmer without a proper key.

Dave

[redcane]

In that case we are on exactly the same page in regards to why the SENC data would need to be encrypted, and it will just to prevent data leakage and not for any technical reason.

I'm guessing that with a hardware device you can probably store the data unencrypted on permanent storage since you can't get access to the data (without opening the device at least). In the software only side we probably have more onerous requirements.

This leads to another problem. How can the S-63 plugin ensure that OpenCPN is encrypting it's SENCs? From a technical sense this is a difficult problem. I think it means the plugin needs to verify it's running with a known build of OpenCPN that hasn't been tampered with. I guess this also depends on the nature of the requirements in the manufacturer key agreement. It might require that the OpenCPN build is signed in some way before the plugin supplies it unencrypted data.

On a side note I'm not really familiar with what formats are in use currently by recreational sailors and OpenCPN users. From a local perspective it seems the Australian Hydrographic office only makes it's data available in S-63, and it appears that the marine chart industry in general is heading this way. This to me is why S63 capability is important. Perhaps there are other options targeted more at the recreational market I'm not aware of?

[bcn]

The relevant IHO documents with respect of s-63 can be found here:
Downloads for IHO Member States and Subscribers

Tons of paper but this normal for specs.

In relation with the technical requierements, the Hardware Manufacturing/OEM agreement sheds some light -
http://www.iho.int/iho_pubs/standard...ent_v1.1.1.doc

(An EPS would be in our case the PC running OpenCPN)
It states:

The Company must:
2.5 Protect the data contained within the SENC so that it is:

Copied and write protected within the EPS (i.e. the SENC becomes useless if copied from one EPS to any other EPS, or if the SENC is amended)

and/or

Stored in a way that makes it very hard for a user to determine its contents.

2.6 Not provide any information to Users, whether as part of the EPS, system documentation, or otherwise, concerning where the temporary unprotected ENC file is located within the EPS.

2.7 Not provide to Users a SENC to ENC conversion programme that exports ENCs originally secured by the S-63 Data Protection Scheme, either in written form or incorporated into the system, or in any way suggest to Users how such a programme could be constructed.

2.8 Provide a secure form of unique identification for each unit of the EPS manufactured. This “Hardware ID”, as defined in IHO S-63, must not be produced using sequential numbers. The “Hardware ID” must be stored within the EPS in a secure way.

2.9 Keep a register of all Hardware IDs and User Permits (both as defined in IHO S-63) created, and if requested by the SA supply a copy of the register to the SA within 15 working days.

2.10 Not provide any information to Users, whether as part of the EPS, system documentation, or otherwise, concerning the disclosure or manipulation of the “Hardware ID” information.

2.11 Produce a User Permit (as defined in IHO S-63) for each applicable EPS manufactured, and ensure this User Permit is freely available to the User.

--------------------------------------------------------------------

This is the part concerning the copyright/encryption part.
Other aspects cover the authentification and update of the documents.

The developers might comment if this is similar to the protection schemes used for the BSB or NV pluins.
It seems(*) clear that the individual licence is bound to one machine or ENC system

In the download there is the test suite. An OEM has to come recommended by an Hidrogrphic Institute to get a Manufacturer licence.

PC based products (not ECDIS conform) that offer S-63 support (user of this products are invited to comment about their experiences with respect to S-63 integration, update, portability etc.):

Rose Points Costal Explorer
Polar View (Win, MAC, Linux)
(*) PolarView allows for five installation keys, would be interesting to know how this deals with the S-63 restrictions - (Brak?)
Fugawi
MaxSea/Furuno (they have ECDIS stuff too)

From what I've seen at the sites of Coastal Explorer and Polar Navy I'm not sure if they have an OEM agreement/Manufacturer key.

In both cases the S-63 material is aquired through ChartWorld and it seems that the licence keys are generated there.
They have two sources of S-63 material:
- Navionics charts in S-63 format
- S-63 charts from UKHO (Admiralty charts)

In theory you have to be able to buy from other sources as well, Australia was mentioned in the Polarview forum.

Resuming....

There are at least three different aspects

- a plug-in that conforms to the rules established for S-63 -> tecnical side
- a company to sign the OEM agreement with the IHO and administering the keys and licences
- licencing charts from the HOs or their sub-licencees

Hubert

[redcane]

Originally Posted by bcn

An OEM has to come recommended by an Hidrogrphic Institute to get a Manufacturer licence.

That could be an issue.

Originally Posted by bcn

(*) PolarView allows for five installation keys, would be interesting to know how this deals with the S-63 restrictions - (Brak?)

I'm guessing you can install PolarView five times, but have to purchase the maps for each installation. Confirmation would be good. Given the requirement to keep a register of all hardware IDs and user permits this could be tied to the license key, or the software would have to "phone home" to get a hardware id.

[K_V_B]

Originally Posted by bcn

2.6 Not provide any information to Users, whether as part of the EPS, system documentation, or otherwise, concerning where the temporary unprotected ENC file is located within the EPS.

I wonder how this could be done in open source. Open source is by definition fully documented. You can't keep details like where files are stored away from the user.

[bcn]

Originally Posted by K_V_B

I wonder how this could be done in open source. Open source is by definition fully documented. You can't keep details like where files are stored away from the user.

Well, a binary - not published source - plug-in can do so. It unpacks the received chart data, decrypts, validates, and puts them into the SENC format of OpenCPN. That should satisfy the requirement.

And in this context the question how the existing two plug-ins (BVB and NV) are handling it.

[redcane]

The other option is not to use any temporary files and only store it in memory. This might mean using a lot of memory, but that could be swap space/virtual memory. It might not work with OpenCPN's architecture but you could also just access the required parts of the file on the fly, perhaps by caching an index of required file offsets.

We'd have to make sure the SENCs aren't decoded anywhere during OpenCPNs operation. Whilst it's likely trivial for someone to modify the source and build a version that does do this, I'm not sure that counts as suggesting how a programme to convert SENC to ENCs could be constructed. We just have to be sure we (the organisation signing the agreement) don't mention it, I'm not sure if it would be an issue if people outside the organisation did.

I'm actually surprised by the laxity of this requirement - it does appear they allow you to decrypt the ENCs to disk, even if only temporarily.

[bcn]

The need of a Godfather...

Originally Posted by redcane

That could be an issue.

From the Annex A of the S-63 Document.

S-63 Annex A
Data Server Certificate Request Procedure
(This refers if you want to become a chart provider)

2.3 Non-Hydrographic Offices and Non-RENC Organisations
Other commercial organisations who wish to operate as Data Servers and encrypt and digitally sign ENC information compliant with the protection scheme can apply for a Data Server Certificate. Such organisations must get a Data Server, already a member of the protection scheme, to endorse the request and complete part II of the form. It is assumed that the Data Server providing the ENC data to the commercial organisation will endorse the request
-----------------------------------------------------------------------
S-63 Annex B
Manufacturer Information Request Procedure

covers the procedure to get a Manufacturer ID/Key --> required for an OpenCPN plug-in

This consists basicly to perform correctly the test suite (and that you comply with the obligations as defined in the OEM Agreement).
------------------------------------------------------------------------
So that should not be an issue

[bcn]

Originally Posted by redcane

I'm actually surprised by the laxity of this requirement - it does appear they allow you to decrypt the ENCs to disk, even if only temporarily.

The question is just to comply with the requirements - it would be interesting to have a look how Polarview and Coastal explorer are handling this.

The encryption and authentification scheme is some years old. But a discussion if you can bypass it can not be our issue.
The encrytion is done with Blowfish/40bit keys. Blowfish is not broken as far as I know. However a 40bit key you can knock-off today with brutal force approaches, even not being a 3-letter organisation. But again, this is not the point.

Just the question: what has to be done to have a solution that is legal and accepted in the light of the IHO requirements.

[redcane]

From what I've understood so far a trivial solution to their requirements for a closed source product is to decrypt the S63 into S57 at startup into a temp folder, as long as there is no documentation of where that temporary folder is located (and being closed source you can't check the source). You could then just operate from the temporary files. In fact that paragraph 2.6 seems to assume you *will* have an unencrypted temporary ENC (this may only be during an import to the software's own internal SENC format). I wonder if you could use an open source algorithm to generate random folders and then only use one, making it hard to determine which was in use? It might also be worth asking if they would consider the source code itself as documentation.

It would be easy to determine where the files are being placed with commercial software by using a process monitor utility, and this might give some insight into how polarview and coastal explorer are operating.

Thanks for looking into that in such detail. I will try to find time this weekend to make some enquiries.

[bcn]

We will need some advice from Dave as well, as he states that the OpenCPN SENC is portable - that would imply included decyphered S-63 info too.

As the S-63 information has to be bonded to one EPS/PC this might imply that you need two concurrend SENC databases at the same time.
Is this feasible?
Or to have encrypted parts of the SENC that have to be uncrypted on-the-fly using the individual key of the EPS/PC. Those parts would then not be usuable on a different machine.

Encryption makes your life easier

[bcn]

An interactive map of existing S-63 charts can be found here:
http://www.iho-wms.net/index_remote.htm

[bdbcat]

Folks...

A technical view:

The OCPN PlugIn scheme comprehends the idea of PlugIn chart types. This is how we do licensed/encrypted NVC Charts today. The interface to the PlugIn chart is essentially a firewall. The PlugIn is responsible for all of its own data management and actual rendering on call by the OCPN core.

The definition can be found in ocpn_plugin.h

class DECL_EXP PlugInChartBase : public wxObject

The class contains methods like:

            virtual int Init( const wxString& full_path, int init_flags );
            virtual wxBitmap &RenderRegionView(const PlugIn_ViewPort& VPoint, const wxRegion &Region);
.
.
.

The PlugIn itself can be a closed source binary blob, as is the current NVC PlugIn.

Regarding SENCs, we can consider that the encrypted SENCs on disk belong to the PlugIn, and might even logically consider them to be cached artifacts of the chart rendering process. In other words, SENCs are not strictly necessary for chart rendering, they just make it much faster after the first chart load. The PlugIn could easily promise not to leave a decrypted copy of anything like a SENC anywhere on the system.

I think that the specs discuss SENCs deeply because some vendors deliver charts to end-users in the form of SENCs already built for the particular hardware involved. Since we don't do anything like this, we can bury the notion of SENCs within the PlugIn, even change the name of SENC to something like "OCPN Object Cache", or OOC file.

If we can make progress on the business side here, I can commit to building an S57 PlugIn (derived from PlugInChartBase) that might be easily adapted to S63 by including a stubbed-out encryption layer in the PlugIn code. Just to prove the concept, you understand....

Progress
Dave

[bcn]

Originally Posted by bcn

There are at least three different aspects

- a plug-in that conforms to the rules established for S-63 -> tecnical side
- a company to sign the OEM agreement with the IHO and administering the keys and licences
- licencing charts from the HOs or their sub-licencees

Great Dave, so point 1 seems getting closer to solutions.

Point two: the OEM to issue Userpermits wit his Manufacturer-ID and Manufacturer-key

The OEM agreement is quite simple and straightforward.
It does not exclude that even a single person can act as a manufacturer.
That is something to be discussed with the IHB and might take some time.
Will tackle this next week.

The OEM needs some infrastructure to generate and administer the keys.
Documentation and records are to be kept in order to comply with the OEM-agreement. Not a favorite task, I admit.

At this moment for me it is not clear if this administrative task really has to be run as a business. I would not exclude that this part can done as a community effort. Of course one person has to expose itself and sign with the IHB.
Btw.: There are no penalties foreseen in the agreement - just to exclude you revoking your keys if you make nonsense.

Why not a business? Well, let's see numbers.
How many plug-ins are going to be installed per year?
100? 1000? 10000?
How much should one charge for it?
5€/6$? 10€/12$?
So with 10€ and 1000 installations we have a gross income of 10.000€.
You have to set up the business, make your tax stuff etc.
Running it as a company response times and support are getting higher on the ranking, so perhaps there are two or three people involved to make it run smoothly for the users. And 1000 licences/yr is a lot of stuff.

We might come also to the conclusion that we can start on a "non-business" base and if it is getting really busy to switch.

Point 3: Charts

There is no reason that this part has to be run by the OEM. With the Userpermits issued for the OpenCPN installed plug-ins by the OEM it will be possible to buy S-63 charts from resellers or the different HOs, entities that distribute S-63 charts.

It seems that the barriers to be reseller are higher and at this given moment there are very few HOs that are flexible enough to deal with exoctic guys (for them) like us.
They want to see cash and are pushed by their governments to be entities that don't cost money. Like it or not, that is at least here in Europe the actual situation.

That does not exclude that once the S-63 plug-in is working and OpenCPN has its manufacturing key one can come back to this issue. Or somebody works on it in parallel.

At least with the plug-in we will have a way for people to get (official) charts. On a complete legal way.

Well, not a paradise but the outlook is not that bad. Looks feasible for me.

Hubert

[SamsonovAnton]

I'm currently developing a private S63/S57-to-SENC converter, and your enthusiasm seems quite naïve to me.

First of all, the very basis of S-63 protection scheme lies around the fact that the user has no direct access to the internals of his/her ECDIS: not only the ENC/SENC files are not available to the user, but the Hardware-ID as well, because the latter serves as the encryption-decryption key. To protect the real Hardware-ID from the user, it is encrypted with the manufacturer key (that is also known to the chart supplier). No matter whether the ECDIS is open-sourced or not — if the user is able to access the file system with plain-text data, then the information can no longer be considered secure. Even if that sensitive data on disk is also encrypted, but the user is able to run arbitrary processes in the operating system, it also means that those keys can be retrieved from memory with a debugger. It would require a state-of-the-art code obfuscation and memory management techniques (those usually used by top-grade cryptography and copy-protection software) to keep the secret data undisclosed.

One may argue that there already exist several software-only products that offer S-63 functionality. Well, I'm not familiar with it, but I'm pretty sure that their creators did put significant effort to comply with the protection requirements as much as it's technically possible.

But there is also a possibility that IHO does not actually conduct implementation audits against the manufacturers, or does it in a very formal way with no real checks. I'm supposing this because recently I was about to apply for the manufacturer key myself, but was scared by that potentially thorough certification. Then I looked into the list of authorized manufacturers and, to my surprise, found my customer there. I asked them whether they could generate a Hardware-ID for testing purposes, and the contact person replied that he was not aware of this at at all, even though he is the main chart-related developer in that company. He was unable to find anybody that really managed their relationships with IHO (the listed “contact person” is actually a director general), so I concluded that no real audit was ever made on them.

As an alternative to a pure-software solution, you may consider using a specialized hardware token with some form of task offloading.

• It may simply store the Hardware-ID in a secure form.
• It may perform cyphering, as Blowfish is a very fast algorithm (the only drawback for embedded devices is that it requires more than 4KB of memory).
• It may even offload more complex tasks, depending on the available computing power and USB bandwidth.

The latter is the most DRM-friendly, but it may require frequent firmware updates, especially in the beginning.


Second, the data layout for the S-63 is significantly more sophisticated than for the S-57. As far as I understand, S-63 was adopted from a proprietary protection system, which was enhanced a little, but seen virtually no development since then. With S-57, basicly all you need to handle are the *.000 charts themselves. To properly manage S-63 datasets, you should be aware of:

• CATALOG.031 — S-57 chart catalog, expanded with additional meta-data so that the charts do not need to be decrypted just for reading their headers; by the way, GDAL does not support catalog files;
• user permit — encoded Hardware-ID string to display to the user so he/she could order charts from a supplier;
• PRODUCTS.LST — global meta-data about chart coverage, revisions and formats;
• PERMIT.TXT — decryption keys and expiration terms;
• SERIAL.ENC — chart serialization meta-data;
• STATUS.LST — distribution media meta-data;
• MEDIA.TXT — distribution media listing;
• IHO.CRT — root signing certificate (X.509) for verification of chart integrity;
• *.PQG — digital signature (SHA-1 + DSA) for specific chart file;
• license.LIC and README.TXT — to display licensing data and other notes (optional).

To make things worse, the S-63 doesn't define any specific data exchange protocols, so any supplier (data server) may provide data of arbitrary layout. If one doesn't want to stick to a sole supplier, then the software should be able to handle an arbitrary combination of datasets — from various suppliers, perhaps partially duplicating each other in respect to cells (charts) but with different editions / revisions and different expiration terms. To be honest, I myself got stuck with all those complications, especially without a real-world datasets to learn upon. So I warn you to not underestimate the burden of implementation.

As for the S-64 demo dataset, it's practically unusable:

• it serves for conducting implementation checks of the most critical functionality, thus contains very basic samples;
• its encrypted part currently contains incorrect CATALOG.031 files for almost half of the samples, perhaps because those files were artificially altered by a non-standard software;
• it provides a root certificate that expired on 2013-08-29; funny enough, some real-world datasets, such as Australian, are still bundled with the same certificate, too.

When I reported the last two issues to the authors, saying that a correctly implemented application would never pass the S-64 checks as they are incorrect / outdated, they replied that they are already aware of this and are reviewing the dataset, but still no change. Actually, it is the same version that was created back in 2007–2009. I wonder how the certifications were conducted all this time since then.

Another example of IHO's conservatism is duplicate agency codes in the S-62 list. About a year ago, I tried to contact IHO to report on having the same code 16038 for two distinct producers. First tried @ihb.mc, then @registry.iho.int — both were inaccessible due to the misconfigured mail server. Finally was able to reach the Big guy who forwarded the message, but, as you can see for yourself, the error is still there in the database.

So don't be very hopeful about IHO being open and flexible enough to deal with exotic guys (for them) like us.