Question About the Plugin Interface

[PjotrC]

Having now the possibility to set a different planned speed per each route leg, I would like to match this to a GRIB weather forecast for more comprehensive route planning.
This means I need to access at the same time:

- route information and its Properties Dialog (I don't want to grow the user interface)
- Tides and Currents data (simple query/response with TC Manager)
- GRIB data (as a minimum query with time+position and get response with wind forecast)

In the monolithic program this would not be difficult...

I think I need almost 100% of GRIB Plugin functionality - find, select and open the file, parse contents etc. Also, while planning a passage, it would always be useful to have all the GRIB data rendered on screen for having a broader picture.

The current Plugin Architecture positions plugins mainly as drawing tools, with - understandably - very limited data flow across the interface.
Respecting this, I am wondering how to proceed, to avoid code duplication and to stay within the prescribed interface.

In future this might occur again:

A DSC Radio Integration Plugin might want to create waypoints and routes.
An AIS Plugin might share the MMSI database with DSC Radio Plugin
A Weather Routing Plugin might want to create routes and access GRIB Records.
A Chart Markup Plugin might want to access all NavObjects.
All Plugins would save/restore config options.

The global application config object is available for plugins, but in order to share data via this structure some protocol is needed e.g. for passing identities and function requests.

In principle, I could coin a proprietary NMEA sentence for communication from main program to plugin. This would be largely sufficient (quite often the existing sentences will do), and the mechanism is already there, but one-way only. Perhaps Plugins could - if they wished - send NMEA sentences back into a common stream for input to the main program? The plugins would then behave similarly to external devices, which in fact they might sometimes handle. This is the "minimum change" method I see, but I haven't yet looked at implementation.

Any hints as to possible data sharing?

Piotr

PS. My current perspective is a simple extension of Route Properties to evaluate one given, fixed Route. Full Weather Routing will deserve packaging into a separate Plugin, but will present similar issues.

[bdbcat]

Piotr....

You bring up some very valid points. Better communication between PlugIns and host (or PlugIn to PlugIn) are highly to be desired.

I am open to any ideas to extend the PlugIn API, including:

1. a bidirectional message channel, with a private language (NMEA style?). Maybe implemented as network socket connection for portability and extensibility.
2. Continued increases in the number and quality of exposed host data structures, like routes, waypoints, AIS info, etc.

Right now is an excellent time to propose API extensions, since we are just starting a Beta cycle. My plan has been to respond to requests by PlugIn developers liberally and quickly, as opposed to trying to anticipate all future needs in the API by specifying in a vacuum.

To be specific on your current project, I think it makes more sense to modify the existing GRIB PlugIn to accept querys, than it would be to include all the GRIB processing code in your new PlugIn. Seems much less effort, and improves the existing code for other future uses.

Anyway, all proposals in this area, however involved they may become, are entertained. We need only preserve backward compatibility.

Dave

[nohal]

Hi,
I would say navobjects are clearly the highest priority. They need some work even in the core to support GPX extensions so that we can interoperate better with the outer world and maybe also allow plug-ins to store data they may produce in a transparent and universal way. For example the measured environmental data could be stored with the trackpoints, etc.

Piping for multidirectional communication would be also nice but for me needed later.

Pavel

[bdbcat]

Gentle Developers....

Time the revive this thread.

As we move toward a 2.6 Beta program start, I'd like to improve/extend the PlugIn API to satisfy some of the concerns expressed here and elsewhere.

Where to begin:
Stream of consciousness.....

What we need:
1. Core-to-PlugIn channel.
2. PlugIn to sibling PlugIn channel.

Piotrs approach is valid for exploring the interaction of two PlugIns, and I congratulate him on his work in this area.

However, is not very extensible. It seems to require a priori that the core know about all possible sharable data values that any future PlugIn may wish to publish. So, code like this in ocpn_plugin.h

      // Plugin Data Exchange Protocol definitions

#define     GRIB_LAT    1

seems not the best way.

Other approaches:

1. Some sort of higher level protocol, wherein the pluginmanager queries each plugin to get a list of variable that the particular plugin is willing to share. The list includes identifier constants, as well as maybe a human readable string that could be parsed by all other interested parties?

unimplemented example:

grib_pi publishes a structure (via html, or JSON?)
int Identifier = GRIB_LAT;
char[] description = "Latitude of interest"
int data_type_id = DATA_TYPE_DOUBLE

piotr's plugin queries the pluginmanager to get, in series, all the data available from all sibling plugins. If it sees something it needs, by context of "desciption", then it can ask for it by Identifier name. If it cannot find what it needs, too bad....

This could be extended into a Query/Response sort of dialog between PlugIns, with the pluginmanager acting as a conduit for opaque data.

This will of course require that all PlugIns remain forever downward compatible in their published interface.

So, for example, if PlugIn B needs PlugIn A,Version 1.1, then PlugIn A version 1.2 should (must?) support the same interface as A v1.1. Otherwise, B breaks when A is upgraded....

I searched the Web for something like this in available code, but nothing found....

Other ideas:

A VARIANT approach.
I have never really needed to use VARIANTs beyond simple pedantic examples, so could use some discussion along these lines. Anyone got ideas?

...or...
an SQL based approach?

Yet more ideas solicited. Surely we are not the first to have to implement this sort of architecture.....

Thinking cap strapped on, smoke emitting....
Dave

[PjotrC]

Dave,

I was about to start rewriting today the interface used in GRIB Route Planner, but for a slightly different reason.

The need for the core to know the constants identifying data items sent to/from plugins is not as limiting as it indeed looks...

1. Of course the core needs to recognize any value it will handle itself.
2. For the values the core does not want to know, it is sufficient for IDs not to conflict.
3. The identifiers need only be unique, no other encoding characteristics.
4. A conflict may arise if two or more plugin developers will simultaneously extend the PDX list in their environments.
5. This is serious only if we expect more than one new plugin appearing (in final form) per day
6. The final integration indeed requires "approved" addition of the identifiers to the ever growing global list.
7. But the compiled interface does not change anymore...

What I started using was the most basic thing I could think of, since I wanted to move on with the functionality.
Now I would replace the single-value parameter passing with a - say - triple, plus - optional - addressee/originator identification.
The reason for triples is to make at least some sequences indivisible.
The reason for identification is that there may be e.g. more than one plugin responding with a wind forecast (e.g. a current GRIB, a DWD RTTY forecast, and a stochastic forecast based on COGOW...), and I might wish to choose the answer I need.



Piotr

[sinbad7]

Learned gentlemen!

Whilst totally unqualified to join the finer points of a new PlugIn structure,I would emphasize the importance of NMEA compatibility.

I am hoping someone will create an NMEA based PlugIn to automatically poll the Globalstar SPOT database to display the locations of SPOT equipped vessels.

The Globalstar SPOT database provides data in .xml format which need translation to NMEA format. Although presently internet dependent,Globalstar with its duplex system has the ability to accept short burst data polling via satellite,which would make the Globalstar SPOT location system both unique and affordable.

Tore

[sinbad7]

re #6

Access to world-wide AIS and SPOT data,including own ship if so equipped,is of course available today provided you have internet access via satellite connection to Iridium, or Globalstar which could perhaps make a special rate for the short burst data collection.

Unless you have an AIS transponder,(most leisure vessels have just a B type AIS receiver),you only have access to AIS data over the internet provided there is an AIS land station covering your area and range.

A plug-in to internet via a satellite connection would allow the reception of BOTH AIS as well as SPOT equipped vessels.

Tore

[bcn]

Originally Posted by sinbad7

re #6


Unless you have an AIS transponder,(most leisure vessels have just a B type AIS receiver),you only have access to AIS data over the internet provided there is an AIS land station covering your area and range.

Tore

Tore,

a typo?
An AIS receiver receives all type of AIS traffic. Only transponders are class A or B where the latter is thought for leisure crafts/non SOLAS ships.
And prices are going down. A class B transponder you can get in Europe for 400€ (incl.tax) nowadays . Receivers are at a 200€ level.

Cheers

Hubert

[sinbad7]

Hubert..

Not sure what 'typo' you are referring to?
Of course an AIS receiver receives all types of traffic.

I was referring to the fact that you only have access (receiving) AIS data from the
internet provided there is and AIS (land) receiving station covering your area and within range..

Tore

[bdbcat]

Pjotr....

Thanks for the feedback.


OK, I see your logic. We just need non-colliding identifiers. Maybe we can invent something based on PlugIn name or version, so that there is at least a standard for future work.

Lets see your proposed struct{}; or whatever....

In this scenario, I see the following conversions:

PI author B:
"Ahoy PI author A! I need to have read/write access to variable C within your PlugIn. Would you be willing to release a new version with PDX support for variable C? And while we are talking, how about a new public method D()?"

PI Author A:
"Sure, I'll get right on it...."

Not a bad strategy, if the authors stick around. I think they will, in general, or if not, then Author B can update and release the OpenSource PlugIn A on his own, and the world is a better place.

I'm still kicking around the idea of having the core poll each PlugIn at Init() time, and maintaining a list of identifiers, types, etc. Just as a common list, you understand, not as an arbiter or throttle point.....

One advantage of this is that OCPN_Plugin.h does not get clogged with possibly deprecated, archaic, or (bless me) wrong stuff over time. Also fixes the identifier collision question....

Anyway, since you are the point guy on actually implementing peer-to-peer PlugIn communication, lets see your specific idea. I'm all ears.

Dave

[dacust]

Originally Posted by bdbcat

... I'm all ears.

Dave

Photo required.

[bdbcat]

Ears at play....

[PjotrC]

Attached are some notes on possible scenarios of plugin conversations.

It was just easier to handle this in a plain text file... also, this is not for the faint-hearted...

Piotr

[rooiedirk]

I would like to suggest to keep it as "open" as possible.

Pjotr did mention 4 scenarios, using a max of three sending and three receiving values, but much more could be possible.

I think something like:

bool DoDataExchange ( Description, V1, V2, V3, &A1, &A2, &A3 );

Where: Description is a String. ("GetWindGRIB_Lat_Lon_Time" or "GetWaveGRIB_Lat_Lon_Time" etc,)
V1 to V3 are setting values as Variant(make the to be used as float for lat long, int as mmsi, string etc
A1 to A3 are the answer Values from the plugin aslo as Variant.
Return code True if answerValues are set.

In the plugin you would need a Select(Description) .. case to check of this particular plugin can answer the question in asked in Description.
In the core code you would need also the same Select(Description) .. case statement, so the PI could also request data from core code.

The plugin-engine would send the "DoDataExchange(..)" to all plugins, and th PI will 'select' by Description if it to be anwered or not.
You could have a "GetWindGRIB_Lat_Lon_Time" but also a "GetWindLOGBOOK_Lat_Lon_Time" or a "GetWindDASHBOARD"

The advantage of above is that we will need only one function to get almost endless possibilities.

Apart from above it would also be useful for the PI to be able to send NMEA strings. This could be using the same system if he core code would understand "DoDataExchange(SendAsNMEA, 0,0,0, thenmeaString..)"

Hope I was able to make clear enough still vague ideas.

Dirk

[tom_BigSpeedy]

Hi all,

I just want give you another possible solution on this topic ...

"Professional" industry programs generally use a socalled "middleware"
(a process running in parallel) to connect/link different programs to each other.


The middleware uses specific "connectors" to link up to each application.

The mentioned connectors are simply routines implemented in the code of
the middleware which call each application to query/submit data, be it the core of OpenCPN or any plugin.

The middleware then handles the requests between the applications i.e.
converts input-information from application A (using connector routines
for A), converts it to output for application B (via connector routines for B) and finally returns the result from B to A.

Modern middlewares (at least those which I know) normally use XML data
as input/output and do the conversion from input to output format via XSL
stylesheets (although this is just one solution to grant maximum flexibility
in adoption of the interface).


With that kind of approach, it's possible to
* handle different versions of plugins
--> all you need is a specific connector for each version. The data
conversion inside the middleware guarantees, that you receive the
data (format) expected.
* query for different data sources for grib files
--> the middleware returns a "unknown" if not available
--> the data conversion inside the middleware guarantees that you
always receive data in the correct format

The advantage is, that you only have to implement the connection to ONE application (the middleware process) and don't have to take care of different programming styles in different plugins ...


Thomas