OSGi Manifests for jar files

[JIDE Support]

You can just post it here.

Thanks,

[chrismhopkins]

Alright. I've made a first pass and have run into a pretty big issue. The way you have the jar files currently organized you have the com.jidesoft.plaf (and sub-packages) in multiple jars. This works fine for regular Java but you cannot have multiple bundles (basically an OSGi-fied jar file) exporting the same package in OSGi*. If you want to support OSGi via your standard distribution then you will have to create something like a jide-plaf.jar and pull together all of the com.jidesoft.plaf.* packages into one jar file and then export those packages from a single jar. I don't know what that does to your build environment or process but it will have to be resolved before you can out-of-the-box support OSGi with your current distribution.

If you want, I can email you my Skype or GTalk ID and can chat about this with you too.

* Actually, you CAN have multiple bundles contribute the same package but the contents should be identical and the version numbers should be different. This is not what you are accomplishing by spreading the various com.jidesoft.plaf.* packages around.

- Chris

[JIDE Support]

Even not for OSGi, we are aware of this issue. The problem to have jide-plaf.jar is there will be cyclic dependency. Can OSGi accept cyclic dependency?

Thanks,

[chrismhopkins]

As far as I know, bundles cannot contain cyclic dependencies, no.

Is the fixing of the cyclic dependency something you are working on?

[JIDE Support]

There are no way to fix the cyclic dependency. Component depends on the corresponding ComponentUI and ComponentUI of course also needs to depend on the Component class. The only solution is to bundle all classes into one jide.jar. However if we do that, all customers have to carry a huge jide jar even they don't use all the products it contains.

Thanks,

[chrismhopkins]

Okay. Here's a suggestion then. For each of the created jar files move the com.jidesoft.plaf into another package. For example, for the jide-action.jar create a com.jidesoft.action.plaf package. That way you can continue to have all of the PLAF files bundled with their corresponding components and still maintain the separation of jars for OSGi. I realize that this will not be a backwards compatible change for those that depend on the com.jidesoft.plaf.* so I'm not sure how you feel about that.

Btw, we have been creating a single OSGi JIDE jar by combining the set of required jars in your distribution so that's why we haven't seen this issue. If there's anything I can do to help, please let me know.

Thanks,
Chris

[JIDE Support]

That's probably not an option. What's in your manifest file for the single jide jar. Maybe you can post it here and let users do it themselves. If they want to use OSGi, to ask our user to bundle all jide jars into one jar and include a OSGi compatible OSGi is not a big deal.

Thanks,

[eduardop]

I've been using JIDE in my own OSGi based application (uses Equinox, eclipse implemenation of OSGi). The classloader magic was a bit tricky and I was able to make things work without relying on implemenation specifc (buddy-policy) OSGi extensions. JIDE needs to be available to ALL of the bundles in the framework as well as the root classloader (for swing). I was able to do this by making JIDE jars into Fragment bundles of the System Bundle. Additionally, I had to combine all of the JIDE jars into one jar (hope I'm allowed to do this). This combined jar/bundle then exports ALL of the packages.

Here is the manifest line I needed:

Fragment-Host: system.bundle; extension:=framework

I don't think this works in Apache Felix because they haven't fully implements the OSGi R4 spec whereas Eclipse Equinox has.

@basszero: are you using any look and feel from a third party jar (like JGoodies)? I'm using PlasticXPLookAndFeel and I got some "UIDefaults.getUI() failed: no ComponentUI class for: com.jidesoft.xxx" errors in the ui bundle activation.
What I need to do to make jide work with the plastic LnF in an osgi environment?

Thanks!

[JIDE Support]

Please check if you invoked LookAndFeelFactory#installJideExtension().

Thanks,

[sgr]

Hello all,

I thought I should resurrect this thread as I would like to contribute to the proper OSGification of the JIDE libraries
and hopefully provide some Ant scripts for JIDE to embed in their buid life-cycle in order for us to download the JIDE
libraries as OSGi bundles instead of having to do it manually and support custom scripts.

As chrismhopkins has already noted, the biggest problem is that the com.jidesoft.plaf package (and sub-packages)
are split in multiple jars. This in OSGi jargon is the'split package' problem and is considered a bad practice.
A more in-depth article for those who are interested : Split Packages – An OSGi nightmare

Now in order to properly OSGify the libraries someone must have the following:
1) Access to the source code (which I don't). This is not strictly necessary but it helps due to the split-package problem
2) A good knowledge of the actual code and more specifically the package dependencies between libraries (e.g. does com.jidesoft.plaf package of the jide-action.jar
depends on the com.jidesoft.plaf package of the jide-common.jar?)
3) Being a commercial library product, the ability to decide which packages are actually needed to be exported by each bundle and which can be kept private (i.e. if
the com.jidesoft.plaf package of the jide-action library does not contain any classes that are meant to be used by the other libraries or the end-user code
then there is no reason to export them in the first place OR if we really have to export the packages the ability to decide and rename the conflicting packages, etc.)

So the best fit for taking this task would obviously be JIDESoft but as I understand they currently do not have any OSGi exposure.

I have already started writing some Ant scripts that perform the OSGification of the JIDE libraries using the Bnd tool
but in order to continue I need some direction on what is possible or not.

Having said that, I think that we have the following options:

1) If the split packages are not meant to be used outside the bundle then we can simply not export them
2) If 1 is not a viable a option, then we should consider the package rename option which is my opinion is the best solution. If that can not be done currently,
you could at least plan for it for the next major version and deprecate and discourage their usage until then.
3) If 2 is not a viable a option, then we could fallback to the usage of the'Require-Bundle' header.
This is also considered as bad practice and I am not sure it would actually solve the problem. And another side-effect could be requiring the end-user
apps to also make use of the 'Require-Bundle' header when the import split packages.
4) Last resort (and easiest I guess), would be to make all bundles extend the framework with the usage of Fragment-Host: system.bundle; extension:=framework.
But again this is considered bad practice as it brakes the merits of modularization (i.e. you can not use more than one version of a bundle at the same time)
which is the initial intention for switching to an OSGi framework in the first place.

If anyone has any ideas and/or corrections, etc. I would love to hear them.

@JIDE: Please let me know what you think and what's possible so that I can also provide you with some scripts for initial testing ,etc.

Regards,
Panayiotis Vlissidis

[JIDE Support]

There are some classes under com.jidesoft.plaf under the common layer that should be exported (such as LookAndFeelFactory). Other than those, all subpackages under plaf don't need to exported as long as you don't build L&F yourself. So we will only leave com.jidesoft.plaf export for common jar, and remove all other plaf and plaf subpackage export statement from the manifest file. Please let me know if this change will make it compatible with OSGi.

To really resolve the split package issue, we will have to create a plaf for each product such as com.jidesoft.dock.plaf, com.jidesoft.action.plaf etc. It is doable but will affect many users.

[sgr]

Hello again,

I have just finished creating the scripts for the OSGification.
Sorry for the delay but there's not much time to spare and
the process was not that easy as I initially though it would be.

Methodology:

As already mentioned I have used BND in order to do the OSGification.
Benefits are :
1) Writing BND scripts is rather simple and straightforward and all documentation is provided at the site.
2) It provides an Ant task for automating the build.
3) And most importantly it is more likely to create proper results(especially as new packages
are introduced) compared to write everything by hand.

I have created the scripts in almost the same fashion that you have the Maven scripts in your full distribution.
So the main contents of the attached zip file are:
- bnd - Folder that contains all BND scripts
- osgify.xml - Contains the Ant tasks
- osgify.bat - Windows batch file for running the Ant tasks
- osgify.sh - Linux batch file for running the Ant tasks
- JIDEDemoActivator.java - BundleActivator implementation needed for testing
- exclude-plaf-packages.log - Console output when bundles(other than jidesoft-common) are started without
exporting the com.jidesoft.plaf packages
- split-packages-error.log - Console output when bundles export all of their packages and have the split-package issue
- print-packages.bat - Utility that can be used for inspecting the package imports of a jar per package.
- felix-config.properties - Felix sample configuration


A BND scipt has been created for each JIDE jar that we need to convert to a bundle (except the jide-beaninfo.jar and jide-designer.jar)
which all import some base properties/directives (e.g. jide.version) from a defaults file(jide-defaults.bnd) .
Moreover BND scripts have been created for the following:
- jide-demo.bnd - for converting an archive of the examples code into a bundle that we will need for testing.
NOTE: In order create the jide-demo.jar I have opened the examples project in IntelliJ IDEA and created an
artifact(containing only the examples code). The JIDEDemoActivator source file was also added in the examples/Demo folder
in order for it to be present in the created artifact.
- jide-ultimate.bnd - for converting a custom mega bundle into a bundle.
NOTE: If you examine the Ant script, you will see the target create-jide-ultimate-jar which simply unpacks
the desired jars and jars them again to a single artifact(lib/jide-ultimate.jar).
- libs.bnd - Which wraps all JIDE's dependencies into a single mega bundle that
we also need for testing. The resulting bundle is a Fragment of the framework(using the Fragment-Host: system.bundle; extension:=framework header)
in order to avoid any further Class loading issues, etc.

BUNDLES GENERATION:

1) Make sure the jide-demo.jar is located within the lib folder (and don't forget to add the JIDEDemoActivator mentioned above)
2) Execute the osgify target

The script will output all OSGi bundles in the osgi folder.

TESTING:

Now in order to test everything you need to:

1) Install an OSGi framework. I used Apache Felix
2) Install the desired bundles to the framework.In case of Felix you can simply copy them in the %FELIX_INSTALL_DIR%/bundles dir and
Felix will automatically try to install and start them. NOTE: You might need to delete the felix-cache folder between successive runs.
3) Configure the org.osgi.framework.system.packages.extra if required. You can the felix-config.properties for reference
4) Start-up the framework

TEST CASE 1:

The simplest test case is for the jide-ultimate bundle which does not have the 'split-package' issue as eveyrhitng is packed into a single bundle.
In order to test it, simply install the generated osgi/jide-ultimate.jar, osgi/jide-demo.jar and osgi/libs.jar to the OSgi framework
and start it. If evertyhing is done properly you should see the 'Unauthorized usage...' dialog and the demo should be loaded.

TEST CASE 2:

Another test is to use all generated jide bundles that export all of their packages(i.e. the split-package issue).
In order to test it, simply install all the generated bundles from the osgi folder ,except the jide-ultimate.jar.
This will FAIL to start the JIDE Demo suite throwing errors like:

C:\java\felix-framework-4.0.3>felix

C:\java\felix-framework-4.0.3>java.exe -jar bin/felix.jar
ERROR: Bundle com.jidesoft.demo [7] Error starting file:/C:/java/felix-framework-4.0.3/bundle/jide-demo.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.demo [7]: Unable to resolve 7.0: missing requirement [7.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.action)(version>=3.5.0)(!(version>=4.0.0))))
org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.demo [7]: Unable to resolve 7.0: missing requirement [7.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.action)(version>=3.5.0)(!(version>=4.0.0)))
at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3826)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1868)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1191)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:295)
at java.lang.Thread.run(Unknown Source)
ERROR: Bundle com.jidesoft.gantt [12] Error starting file:/C:/java/felix-framework-4.0.3/bundle/jide-gantt.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.gantt [12]: Unable to resolve 12.0: missing requirement [12.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.grid)(version>=3.5.0)(!(version>=4.0.0))))
org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.gantt [12]: Unable to resolve 12.0: missing requirement [12.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.grid)(version>=3.5.0)(!(version>=4.0.0)))
at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3826)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1868)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1191)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:295)
at java.lang.Thread.run(Unknown Source)
ERROR: Bundle com.jidesoft.plaf [25] Error starting file:/C:/java/felix-framework-4.0.3/bundle/jide-plaf.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.plaf [25]: Unable to resolve 25.0: missing requirement [25.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.action)(version>=3.5.0)(!(version>=4.0.0))))
org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.plaf [25]: Unable to resolve 25.0: missing requirement [25.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.action)(version>=3.5.0)(!(version>=4.0.0)))
at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3826)
at org.apache.felix.framework.Felix.startBundle(Felix.java:1868)
at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1191)
at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:295)
at java.lang.Thread.run(Unknown Source)

This is most probably caused by the fact that more than one bundles export the com.jide.soft.* packages and the
bundles resolver can not choose the proper bundle to use. This could be probably solved by either using the Require-Package
header or by installing the bundles in the correct order. But even then, I am not sure if everything would work properly.

TEST CASE 3:

Another test is to avoid exporting the com.jidesoft.plaf.* packages from all bundles, other than the jide-common.
In order to do that, uncomment from all *.bnd files the #Export-Package: !com.jidesoft.plaf.*, *
by simplyremoving the starting hash and re-run the osgify target to re-generate the bundles.
Now install again all the generated bundles from the osgi folder ,except the jide-ultimate.jar.
Make sure the felix-cache is cleared and start Felix again.

This will again FAIL to start the JIDE Demo suite throwing errors like:

Code: Select all

C:\java\felix-framework-4.0.3>felix

C:\java\felix-framework-4.0.3>java.exe -jar bin/felix.jar
icons/title_buttons_metal.gif
icons/collapsible_pane_metal.gif
icons/collapsible_pane_default.png
icons/collapsible_pane_mask.png
icons/collapsible_pane_default.png
icons/collapsible_pane_mask.png
icons/collapsible_pane_gray.png
icons/collapsible_pane_mask.png
icons/collapsible_pane_blue.png
icons/collapsible_pane_mask.png
icons/collapsible_pane_homestead.png
icons/collapsible_pane_mask.png
icons/collapsible_pane_metallic.png
icons/collapsible_pane_mask.png
icons/title_buttons_windows.gif
icons/collapsible_pane_windows.gif
icons/collapsible_pane_vista.gif
icons/title_buttons_office2007.gif
____________________________
Welcome to Apache Felix Gogo

g! Exception in thread "AWT-EventQueue-0" java.lang.NoClassDefFoundError: com/jidesoft/plaf/StatusBarSeparatorUI
        at com.jidesoft.status.MemoryStatusBarItem.<init>(Unknown Source)
        at JideDemos.createStatusBar(JideDemos.java:177)
        at JideDemos.initAndShowGUI(JideDemos.java:128)
        at JideDemos.access$000(JideDemos.java:31)
        at JideDemos$1.run(JideDemos.java:86)
        at java.awt.event.InvocationEvent.dispatch(Unknown Source)
        at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
        at java.awt.EventQueue.access$200(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.awt.EventQueue$3.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
        at java.awt.EventQueue.dispatchEvent(Unknown Source)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
        at java.awt.EventDispatchThread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: com.jidesoft.plaf.StatusBarSeparatorUI not found by com.jidesoft.common [3]
        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1460)
        at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        at org.apache.felix.framework.BundleWiringImpl.getClassByDelegation(BundleWiringImpl.java:1317)
        at org.apache.felix.framework.BundleWiringImpl.searchImports(BundleWiringImpl.java:1481)
        at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1427)
        at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:72)
        at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1843)
        at java.lang.ClassLoader.loadClass(Unknown Source)
        ... 19 more



I think that there might be a cyclic dependency problem there which I think should not pose a problem as I have
set the DynamicImport-Package header to everything so if the packages were available it should work.
But in this case we hide the packages in the first place, so this is not a viable solution.

To summarize, I think that the only proper and viable solution is to have a plaf for each product (as it has already been mentioned).

Please also read and test the scripts yourself and let me know of any findings. If you are completely
new to OSGi, all you need to know are the basics. A good article ishttp://blog.springsource.org/2008/02/18/creating-osgi-bundles/.
And I also strongly recommend OSGI in Action if you need to get some more in-depth knowledge.

[JIDE Support]

Hi Panayiotis,

Thanks for your scripts and detailed description. Really appreciate it.

After all these discussions, We got a pretty clear picture on OSGi. Would like to propose some conclusion.
1. An ultimate bundle including everything will work
2. Any bundle based on current separate modules will not work because of cycle dependency

If this is the case, we would like to provide a ultimate bundle in later release. As to separate bundles, we have to adjust our package structure to avoid cycle dependency. This will not be considered in the near future. Will keep posting on this thread when we have any progress.

Please let us know if we did any misunderstanding.

Thanks,
--

[sgr]

Yes your conclusion is right.

We are perfectly happy using just the OSGified ultimate bundle and we'll be waiting for it.

One last thing to note though, is that even though you are not planning to change your packages any time soon,
you can still use the BND scripts provided to produce more correct OSGi headers for the individual product MANIFEST.MF files.
If you decide not to, then at the very least you should add the Bundle-SymbolicName(which is a required header as of OSGi Release 4.0) and Bundle-Version.

Thanks again for looking into this.

Regards,
Panayiotis Vlissidis

[chrismhopkins]

I'm happy to see traction on this and looking forward to the single jar with OSGi headers. That is currently what we are doing internally as well so will be happy to just pull from the release instead of building our own.

One thing to consider as well is a means of license validation within OSGi. We are running with Equinox (as opposed to Felix or another OSGi container) and we have written a custom bundle Activator that will initialize the JIDE license on bundle startup. YMMV with other OSGi containers though as to the appropriate mechanism so it's probably best not to add that to the release bundle you are building.

[JIDE Support]

Hi Panayiotis,

Will it be helpful if we just add the correct Bundle-SymbolicName and Bundle-Version into individual product MANIFEST.MF? If it will, we will glad to add them. However if it will not, we would like to leave them as is unless we decide to provide individual bundles.

Thanks,
--

[sgr]

Yes, it would help get rid of the warnings generated by IntelliJ IDEA's OSGi plugin.

Thanks

[JIDE Support]

Got it. Will add these headers into MANIFEST.MF in later release.

Thanks,
--

[JIDE Support]

Just so you know, the first OSGi release of JIDE (3.5.1) has been published on our website (a link next to where you normally download the releases). Bundle headers requested by srg have been added to individual MANIFEST.MF as well. Please check it out. Any feedback will be appreciated.

Thanks,
--

[sgr]

Hello,

Thanks for the update.

I have just downloaded and tested the ultimate jars(normal and debug) from the OSGi release.

There is only one problem: You can NOT import the java.sql package or any other java.* package for that matter and the same goes for the Export-Packages header as well.
And you do NOT have to explicitly import them anyway, as they are automatically provided by the OSGI container to every bundle.

For anyone interested, the workaround for using the bundles is to simply remove the java.sql;resolution:=optional from the MANIFEST.MF file of each bundle.

Here's the error from the Felix container, for reference:

Code: Select all

C:\java\felix-framework-4.0.3>java.exe -jar bin/felix.jar
Auto-deploy install: org.osgi.framework.BundleException: Importing java.* packages not allowed: java.sql
ERROR: Bundle com.jidesoft.demo [1] Error starting file:/C:/java/felix-framework-4.0.3/bundle/jide-demo.jar (org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.demo [1]: Unable to resolve 1.0: missing requirement [1.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.action)(version>=3.5.0)(!(version>=4.0.0))))
org.osgi.framework.BundleException: Unresolved constraint in bundle com.jidesoft.demo [1]: Unable to resolve 1.0: missing requirement [1.0] osgi.wiring.package; (&(osgi.wiring.package=com.jidesoft.action)(version>=3.5.0)(!(version>=4.0.0)))
        at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:3826)
        at org.apache.felix.framework.Felix.startBundle(Felix.java:1868)
        at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1191)
        at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:295)
        at java.lang.Thread.run(Unknown Source)
____________________________
Welcome to Apache Felix Gogo

g! lb
START LEVEL 1
   ID|State      |Level|Name
    0|Active     |    0|System Bundle (4.0.3)
    1|Installed  |    1|JIDE Demo (3.5.0)
    3|Resolved   |    1|JIDE Libs (0.0.0)
    4|Active     |    1|Apache Felix Bundle Repository (1.6.6)
    5|Active     |    1|Apache Felix Gogo Command (0.12.0)
    6|Active     |    1|Apache Felix Gogo Runtime (0.10.0)
    7|Active     |    1|Apache Felix Gogo Shell (0.10.0)
g!


[JIDE Support]

Thanks for the bug report. Have re-posted the OSGi release.

Thanks,
--

[ryan_se]

Seems that the pom has gone missing on jide-ultimate for releases after 3.5.9. Is this intentional?