Coverage for gws-app/gws/plugin/alkis/data/__init__.py: 100%

1"""ALKIS data: reading, indexing, searching and exporting. 

2 

3This package holds the data layer of the ALKIS plugin. It reads ALKIS source 

4tables from a PostgreSQL database, builds a compact search index from them in 

5a separate schema, runs Flurstueck (parcel) and address searches against that 

6index, and exports search results to files. The ``alkis`` action and the 

7``gws alkis`` CLI commands are built on top of it. 

8 

9Submodules: 

10 

11- ``types``: the data model. ALKIS entities (``Flurstueck``, ``Lage``, 

12 ``Gebaeude``, ``Buchungsblatt``, ``Person``, ``Part`` and so on), their 

13 records, ``EnumPair`` code/text values, query objects 

14 (``FlurstueckQuery``, ``AdresseQuery`` and their options), the 

15 ``IndexStatus`` and the ``Reader`` interface for source data. 

16- ``norbit``: a ``Reader`` for source tables written by the norBIT ALKIS 

17 import. 

18- ``gid``: the GeoInfoDok schema used by the reader and the indexer, 

19 ``geo_info_dok.gid6`` or ``geo_info_dok.gid7`` depending on 

20 ``GWS_ALKIS_GID_VERSION`` (default ``7``). 

21- ``indexer``: builds the index. It reads all relevant ALKIS object types 

22 through a ``Reader``, links them together in memory and writes the result 

23 into the index tables. 

24- ``index``: the index node. It defines the index tables, reports the index 

25 status, finds Flurstuecke and addresses and loads them with their related 

26 data. It also holds helpers to serialize objects to JSON, to normalize 

27 search strings and to flatten a Flurstueck into rows for export. The index 

28 node is created by the ``alkis`` action, which passes the database, 

29 ``crs``, ``schema``, ``excludeGemarkung`` and ``gemarkungFilter`` options. 

30 If ``gemarkungFilter`` is set, searches and the street list are restricted 

31 to these Gemarkung codes. 

32- ``exporter``: a configurable exporter that writes Flurstuecke to CSV or 

33 GeoJSON, using models whose field names are flat keys 

34 (see ``index.flatten_fs``). If no models are configured, a default model 

35 with basic Flurstueck fields is used. 

36- ``geo_info_dok``: Python modules generated from the GeoInfoDok schema 

37 (``gid6``, ``gid7``) and the generator that creates them. 

38 

39Design: 

40 

41Every ALKIS object is represented by an *entity* with a list of *records* 

42(``recs``), one per version in the object's life span, sorted by start date. 

43A record is historic when its life span has ended; an entity is historic when 

44all of its records are. Code list values are stored as ``EnumPair`` objects 

45holding both the code and the display text. 

46 

47The indexer works in phases: places (Land, Kreis, Gemeinde, Gemarkung and so 

48on), Buchung (Buchungsblatt, Buchungsstelle, Namensnummer, Person, 

49Anschrift), Lage with Gebaeude, Flurstueck data, Parts (Nutzung, Festlegung, 

50Bewertung, computed by intersecting the source geometries with the 

51Flurstuecke) and finally the flat search tables. Each object is stored as a 

52JSONB document; the flat ``index*`` tables hold the searchable columns and 

53refer to Flurstuecke by their uid. Index tables are named 

54``alkis_<VERSION>_<table>`` and live in the configured index schema. 

55 

56A search selects matching Flurstueck uids from the flat tables, then loads the 

57JSONB documents and attaches related data depending on the requested display 

58themes, dropping historic objects unless history display is requested. 

59 

60Example:: 

61 

62 from gws.plugin.alkis.data import index, indexer, types as dt 

63 

64 # build the index from the source schema (ix is an ``index.Object``) 

65 indexer.run(ix, 'public', with_force=True) 

66 

67 # find a Flurstueck with its address and buildings 

68 q = dt.FlurstueckQuery( 

69 gemarkungCode='1234', 

70 flurnummer='5', 

71 zaehler='12', 

72 options=dt.FlurstueckQueryOptions( 

73 displayThemes=[dt.DisplayTheme.lage, dt.DisplayTheme.gebaeude], 

74 ), 

75 ) 

76 for fs in ix.find_flurstueck(q): 

77 print(fs.fsnummer, [la.recs[-1].strasse for la in fs.lageList]) 

78 

79Example:: 

80 

81 actions+ { 

82 type "alkis" 

83 dbUid "DB_ALKIS" 

84 dataSchema "public" 

85 indexSchema "gws8" 

86 crs 25832 

87 ui { useExport true } 

88 exporters+ { 

89 type "csv" 

90 title "Basisdaten (CSV)" 

91 models+ { 

92 fields [ 

93 { type "text" name "fs_flurstueckskennzeichen" title "Kennzeichen" } 

94 { type "float" name "fs_recs_amtlicheFlaeche" title "Flaeche" } 

95 ] 

96 } 

97 } 

98 } 

99""" 

100 

101from . import index, types