State of GeoServer - FOSS4G 2023 Edition

GeoServer is a web service for publishing your geospatial data using industry standards for vector, raster and mapping. Choose additional extensions to process data (either in batch or on the fly) and catalog records.

GeoServer is widely used by organizations throughout the world to manage, disseminate and analyze data at scale. GeoServer web services power a number of open source projects like GeoNode and geOrchestra.

This presentation provides an update on our community as well as reviews of the new and noteworthy features for the latest releases. In particular, we will showcase new features landed in 2.22 and 2.23, as well as a preview of what we have in store for 2.24 (to be released in September 2023).

Attend this talk for a cheerful update on what is happening with this popular OSGeo project, whether you are an expert user, a developer, or simply curious what GeoServer can do for you.

Transcript

1. State of GeoServer 2.23
2. Jody Garnett jody.garnett@geocat.net @jodygarnett@fosstodon.org GeoCat Government Geographic Data publishing Andrea Aime Technical Lead andrea.aime@geo-solutions.it @geowolf GeoSolutions Innovative, robust, cost-effective solutions leveraging best-of-breed Open Source products. This presentation is brought to you by
3. GeoServer at a Glance Java Web Application to share and edit geospatial data. Publish data from any major spatial data source using open standards. Core Protocols WMS – maps WFS – vector WFS-T – editing WCS – coverage WMTS – tiles TMS – tiles WMS-C – tiles Extension/community protocols WPS – process CSW – search OGC API - json + rest standards OGC STAC - spatio-temporal asset catalogue
4. GeoServer Team Update 2023 Update
5. Core committers are nominated by their peers and trusted with the ability to approve pull-requests. We also support community commit access by request (often to work on a specific research and development topic): • 23 committers Project Steering Committee serves to guide the project drawing from the various groups with a stake in the success of the project. • Alessio Fabiani • Andrea Aime • Ian Turton • Jody Garnett • Jukka Rahkonen • Kevin Smith • Nuno Oliveira • Simone Giannecchini • Torben Barsballe GeoServer Team
6. GeoServer maintains a list service providers: • Core Contributors Ongoing commitment to the project devoting resources to security fixes, releases and maintenance activities. • Experienced Providers Successfully contribute functionality to the project on behalf of their customers. • Additional Services Providers Provide training, setup and integration support and assistance using GeoServer. (request: please take part in community) GeoServer Service Providers Policy change recognize participation: Core contributors directly contribute to project sustainability and are recognized for: • core, committer, support Experienced providers participate in aspects of the project as community members: • development, coding, documentation, outreach, translation, training, service, product
7. • Transition from Boundless → Completed! ◦ geoserver.org (GitHub Pages) geoserver.org/blog ◦ github.com/geoserver (GitHub) ◦ blog.geoserver.org (github) ◦ repo.osgeo.org (OSGeo) ◦ domain names (OSGeo) ◦ downloads (SourceForge) ◦ email (SourceForge) ◦ build.geoserver.org (GeoSolutions) ◦ web map (GeoSolutions) ◦ docs.geoserver.org (OSGeo) • New ◦ docker (OSGeo Nexus) GeoServer Infrastructure
8. experiment Community modules GeoServer maintains a “community space” for experiments and new developers to work. • Developers propose a community module idea and request commit access • We make sure they compile! • May be proposed as an extension when both the author and functionality is ready Spot a community module of interest, reach out to see how you can help. Graduating extension • (not yet) Outgoing Community • ows-simulate • nsg-wmts • wmts-styles • teradata • importer-fgdb • saml • wms-eo • geostyler Incoming Community • proxy-base • webp-wms-output • spatialJSON-wfs-output Downgraded extension • xslt • imagemap supported end-of-life un-supported
9. GeoServer Releases 2023 Update
10. March 2023 September 2023 March 2024 2.22.x 2.23.x 2.23.x Releases covered by this presentation YOU ARE HERE 2.24.x 2.24.x September 2024 2.24.x Java 11 minimum! Last Java 8 release
11. Are you using a older version? Upgrade! • Easier to get answers. User list and stack exchange typically cover only supported versions ◦ In September 2023 “supported” means 2.24.x and 2.23.x ◦ In March 2024 “supported” will mean 2.24.x and 2.25.x • Security fixes added to supported versions only • Please upgrade your GeoServer installations! Photo by SpaceX on Unsplash
12. Upgrade? What’s in it for me? • Much! • Check what’s new in 2.22, 2.23 and incoming in 2.24 • Check the bottom of each slide to see who sponsored a certain feature, who implemented it, and what version contains it • This icon marks activities done without any sponsoring Version Author Sponsor ♥
13. Distribution
14. • OGC API Code sprint activity ◦ Goal to get feedback on the new ogcapi services under development ◦ Use of osgeo docker nexus Docker for Nightly Builds (with community modules) • Download from build server ◦ 2.24-SNAPSHOT ◦ community modules also 2.24.x Jody Garnett ♥ docker run -it -p 80:8080 \ --env INSTALL_EXTENSIONS=true \ --env STABLE_EXTENSIONS="wps,css" \ --env COMMUNITY_EXTENSIONS="ogcapi-features" \ -t geoserver-docker.osgeo.org/geoserver:2.24.x docker pull \ geoserver-docker.osgeo.org/geoserver:2.24.x
15. Mapping Data making a scene!
16. Improved mosaicking performance - Trying to mosaic several hundreds of images in the same output? - The index can have millions mind, we’re talking hundreds images to build a single GetMap output - Performance and memory usage for this use case have been improved by a couple of orders of magnitude (GeMap built with 300 images) 2.22 Andrea Aime GeoSolutions USGS
17. Improved hyperspectral performance - Hyperspectral images: those having hundreds of bands - Typically stored in band-interleaved structure (one data bank for all pixels in the same tile) - GS is now orders of magnitude faster at reading them 2.22 Andrea Aime GeoSolutions DLR Rows Cols Bands
18. Raster Attribute Table support - .aux.xml sidecar file associating pixel values with various classifications and colors - Generate SLDs out of them and associate with layers - Mimick QGIS RasterAttributeTable plugin - Development in progress, screenshot from QGIS! Community Andrea Aime GeoSolutions NOAA
19. Data sources and formats News and Updates
20. Feature Type Customisation : Description • Previously in 2.21.x ◦ Rename attributes ◦ Change attribute order ◦ Change attribute type ◦ Generate attributes using expressions • Now in 2.23.x ◦ Description for human readable name 2.23 Joseph Miller (GeoSolutions) ?
21. • Supports efficient access to Cloud Optimized GeoTiff (COG) • HTTP/S3/Azure/Google storage • Minimizes reads on blob storage Cloud Optimized GeoTIFF community module Planet GeoSolutions Header Tile Tile Tile Tile Tile Tile Read 1 Read 2 Josh Fix Daniele Romagnoli • Contact GeoSolutions to sponsor: ◦ More blob storage options ◦ More authentication types support ◦ Caching ◦ Other improvements and fixes Community
22. COG for Azure - Cloud Optimized GeoTIFF native support for Azure blob storage - Allows access to authenticated sources - Native access, improved performance - Community module! Community Daniele Romagnoli GeoSolutions TOTAL
23. STAC Datastore and Mosaic • Pull STAC items from a STAC API • Render footprints on WMS • Mosaic images on the fly • Community module! Community Andrea Aime GeoSolutions DLR STAC API STAC datastore COGs Image mosaic World Settlements Footprints, STAC API + COG
24. Vector mosaicking store • Useful to handle thousands of files with similar structure • Database used only to index them • Much cheaper than storing everything in a database (on cloud) • Not suitable if on the fly aggregations are needed • Great if the end user application mostly pulls a few files (one?) at a time Community Joseph Miller GeoSolutions TROO Community module!
25. WMS, WFS and WMTS cascading improvements • Steady stream of fixes from Roar Brænden • New contributor, 50+ tickets closed in the last year ◦ Strong focus on WFS, WMS and WMTS stores ◦ Feedback and improvement of internal “ResourceStore” • Activity in GeoTools that positively reflects in GeoServer • Live link 2.24 Roar Brænden Norwegian Institute for Water Research 2.23
26. Services News and Updates
27. CSW-ISO module graduation • The CSW-ISO module allows to generate ISO metadata records around GeoServer layer and services. This is a rare come back from community (used to be extension already) 2.22 @fileIdentifier.CharacterString=prefixedName identificationInfo.AbstractMD_Identification.citation.CI_Citation.title.CharacterString=title identificationInfo.AbstractMD_Identification.descriptiveKeywords.MD_Keywords.keyword.CharacterString=keywords identificationInfo.AbstractMD_Identification.abstract.CharacterString=abstract $dateStamp.Date= if_then_else ( isNull("metadata.date") , 'Unknown', "metadata.date") <gmd:MD_Metadata> <gmd:fileIdentifier> <gco:CharacterString>CoverageInfoImpl--4a9eec43:132d48aac79:-8000</gco:CharacterString> </gmd:fileIdentifier> <gmd:dateStamp> <gco:Date>Unknown</gco:Date> </gmd:dateStamp> <gmd:identificationInfo> <gmd:MD_DataIdentification> <gmd:extent> … Niels Charlier Scitus -
28. Metadata module graduation • The metadata module adds an editor for extra metadata information (to be used in CSW-ISO) The editor structure can be customized using YAML configs 2.22 Niels Charlier Scitus - attributes: - key: metadata-identifier fieldType: UUID - key: metadata-datestamp label: Date fieldType: DATETIME - key: data-language fieldType: DROPDOWN values: - dut - eng - fre - ger - key: topic-category fieldType: SUGGESTBOX occurrence: REPEAT values: - farming - …
29. • New OGC Standards ◦ Self describing OpenAPI ◦ REST / JSON ◦ Building on from W3C collaboration and STAC / WFS3 progress • Contact Andrea Aime if you are interested in helping with development or funding! Check out dedicated presentation Demystifying OGC APIs with GeoServer: introduction and status of implementation OGC API community module Andrea Aime GeoSolutions GeoSolutions OGC GeoNovum API Common Maps API Coverages API Implementing (as part of TB14/tb15) Looking for volunteers/sponsors Process API Records API Features API Tiles API Styles API STAC API Community CITE compliant!
30. Configuration and Setup
31. New Welcome Page Layout GSIP-202 Welcome Page Layout - Available now in 2.22-M0 Milestone - Inspired by ogc-api community module - Use title and description to make a heading for each service - List capabilities and tools for each section - Change workspace - A welcome page for each workspace - Click to explore virtual web services - Change layer/group - A welcome page specific to a given layer/group - Layer specific services 2.22 Jody Garnett GeoCat GeoServer Enterprise GeoCat
32. Configure Welcome page selectors • Configure for use with large catalogues ◦ GeoServerHomePage.selectionMode ▪ dropdown - auto complete ▪ text - simple text fields ▪ auto - based on response time ◦ GeoServerHomePage.selectionTimeout ◦ GeoServerHomePage.selectionMaxItems When in ‘text’ mode summary does not try and count available layers 2.23 Andrea Aime GeoSolutions GeoSolutions
33. Style format in styles page • Playing around with multiple styling languages? • SLD 1.0, SLD 1.1, CSS, YSLD, MBStyles, oh my • The styles page now indicates the format for each style 2.22 Mohammad Mohiuddin Ahmed ♥
34. Reset of single store/layer via REST API • Currently one can do a system wide “reset” • Clears up feature type caches, connection pools and so on, from the whole GeoServer • Add the ability to perform reset of caches/pools on a single store and on a single layer 2.22 Andrea Aime GeoSolutions GeoSolutions
35. proxy-base-ext • When proxy-base is not powerful enough…. • Allows deeper URL changes, e.g., expose different services on different host-names • Can expand HTTP header placeholders from the proxy itself Community Joseph Miller GeoSolutions DLR Proxy (e.g. nginx) wms.myhost.com wfs.myhost.com GeoServer http://ip/geoserver/wms http://ip/geoserver/wfs Backlinks in responses with the original host and path (e..g, Capabilities)
36. Space, the next frontier
37. Getting off the EPSG assumption • GeoServer 2.24 will be able to handle more CRS authorities • In particular, IAU is being added, which covers planetary CRSs • The work opens the possibility to have more authorities as well (e.g., ESRI, IGNF, NKG, and more) 2.24 Andrea Aime GeoSolutions USGS
38. Mars, original dataset and north polar reprojection 2.24 Andrea Aime GeoSolutions USGS
39. Work ongoing 2.24 Andrea Aime GeoSolutions USGS • Can be tested right now on the developer nightly build • It’s still far from complete! ◦ Capabilities generators ◦ WMS/WFS/WCS/WPS ◦ Retain CRS in GIS output formats ◦ PostGIS/GeoPackage • Aiming at work completion in time for the 2.24.0 release
40. Security Vulnerabilities
41. • Keep exploit details out of issue report • Mark the issue as a vulnerability. • Be prepared to work with Project Steering Committee (PSC) on a solution • Keep in mind PSC members are volunteers and an extensive fix may require fundraising / resources Report via geoserver-security@lists-osgeo.org or github private vulnerability reporting. If you are not in position to communicate in public please consider commercial support, contacting a PSC member, or reaching us via the Open Source Geospatial Foundation at info@osgeo.org. In case you stumble into a vulnerability: Responsible Disclosure
42. • OGC Filter Injection ◦ CVE-2023-25157 (GeoServer) ◦ CVE-2023-25158 (GeoTools) ◦ Archived releases patched on behalf of customers and projects not in a position to upgrade • Ongoing management of dependencies ◦ Vulnerability not often verified, as these reports can be based on automated scan • New: GitHub vulnerability reporting ◦ Secure communication ◦ Automate CVE assignment Security Vulnerabilities 2.23 GeoServer PSC 🔥🔥🔥
43. Control remote HTTP requests (GSIP-218) One of the longest outstanding vulnerability requests has been to limit the HTTP requests made by GeoServer using user provided locations. However some protocols require access access to external web resources … • Config: Check remote location • WMS dynamic SLD • WMS feature portrayal of remote WFS • WPS remote inputs • More? Funding/volunteer required!
44. • H2 is an embedded database we started using many years ago • Convenient, ended up powering some core functionality: ◦ GWC disk quota (by default) ◦ KML super-overlays index • And some non core functionality ◦ Default DB for GeoFence, JDBCConfig (can use external db), WPS JDBC ◦ Index DB for NetCDF/Grib files • Upgrade to H2 version 2 is hard, different binary format • Couple of CVEs against v1, we did not find a way to use them H2 version 1 removal 2.23 GeoSolutions • In GeoServer 2.23.2, removing core usage • Replaced by HSLQDB • Usages in optional plugins will be removed in time (are you interested?) linz.gov.nz
45. But what about CVE-2023-35042? This is a duplicate of the “Jiffle” vulnerability CVE-2022-24816 patched last year. Researched just saw automated attacks come in and stood up an older unpatched GeoServer to see if something would happen. Not sure why it got a new CVE number so we have asked :) Update: Now marked as “disputed” (which is not quite right)
46. Community Building
47. Participation required for Sustainable Open Source Thanks to new release volunteers, very much appreciated: • Gabriel Roldan (camptocamp) • Peter Smythe GeoServer increasingly relies on a small development team: • Big thanks to our core contributors for making this project happen • Experiment: with sponsorship and small contracts? ◦ Tried with Log4j - amount of overhead in fundraising not worth the time commitment • Experiment: “cost recovery” code-sprint model? ◦ Trying with “remove opengis” sprint - helping cover costs for participants
48. • The community really does not really have a “road map” • The core developers are mostly employed in companies providing commercial services for GeoServer, or using it in some hosting solution → customer driven (no other significant source of funding) • The other developers pop up occasionally to provide improvements, fixes and new features they need Roadmap - whatever else you want to push for Community
49. Thanks!