Coverage for gws-app/gws/plugin/qfieldcloud/__init__.py: 100%

1"""QField Cloud plugin. 

2 

3This plugin emulates the QFieldCloud API, so that the QField mobile app can 

4download projects from GWS and synchronize edits back. The QGIS projects are 

5packaged according to the settings made with the QFieldSync QGIS plugin. 

6 

7.. rubric:: Submodules 

8 

9- ``action``: the ``qfieldcloud`` action. It receives API requests and passes each one to a new request handler. 

10- ``action_base``: the base class of the action. It holds the QField projects, caches their capabilities, creates packages (also in background jobs) and manages the package and cache directories. 

11- ``action_handler``: the request handler. It dispatches an API request to a route method, authenticates the client by token, and applies incoming deltas and file uploads. It holds the state of a single request only and accesses persistent data via the action. 

12- ``api``: data classes and enums mirroring the QFieldCloud API objects (``swagger.yaml``), plus a few objects used by the client that are not in the specification. 

13- ``auth``: the ``qfieldcloud`` authorization method, created and registered by the action. It has the fixed uid ``gws.plugin.qfieldcloud.auth``. 

14- ``caps``: reads the QFieldSync project and layer properties from the QGIS project and decides, per layer, whether it is packaged for editing, packaged as a base map or removed. 

15- ``cli``: the ``qfieldcloudPackage`` command, which creates a package into a directory. 

16- ``core``: the configuration and object of a single QField project. 

17- ``delta``: stores the deltas sent by QField with their status and the payload ids they were sent with, in a SQLite database per project. 

18- ``packager``: writes a package (GeoPackage data, base maps, media files, modified QGIS project). 

19- ``patcher``: applies changes ("deltas") and file uploads from QField to the database. 

20 

21.. rubric:: Design 

22 

23The action holds a list of QField projects (``core.QfcProject``), each built on 

24a QGIS project. For each QField project, ``caps.Parser`` creates a ``caps.Caps`` 

25object with the layer and model map; the action caches it until the QGIS project 

26source changes. Packaging and patching both work on these capabilities. 

27 

28The action answers requests under the raw command ``qfieldcloudApi``. A rewrite 

29rule maps a regular address to that command and names the GWS project:: 

30 

31 web.sites+ { 

32 rewriteRules+ { 

33 pattern "^/qfc/(.*)" 

34 target "/_/qfieldcloudApi/projectUid/my_project/$1" 

35 } 

36 } 

37 

38QField users enter the resulting address (``https://example.com/qfc``) as the 

39server and log in with their GWS credentials. The action brings its own 

40authorization method; an auth provider and a session manager must be configured. 

41Requests other than the public status and login routes require an 

42``Authorization: Token ...`` header with the session token. 

43 

44.. rubric:: Configuration 

45 

46The plugin exposes an action of type ``qfieldcloud``. In the action config, you can define multiple ``projects``, each representing a QField project:: 

47 

48 actions+ { 

49 type "qfieldcloud" 

50 

51 projects+ { 

52 title "My Project" 

53 provider.path "/path/to/file.qgs" 

54 } 

55 } 

56 

57.. rubric:: Downloading data 

58 

59Data flow GWS -> QField, also called "packaging". 

60 

61When QField requests a package job, the action runs the packager in a background job. 

62In the given QGIS project, the plugin looks for Postgres layers marked as "offline editable", fetches their data and writes each table into a GeoPackage file. A modified QGIS project file is also created, pointing to the GeoPackage layers instead of the original Postgres layers. Layers marked "remove" are removed from the project, and empty layer groups are dropped. 

63 

64For each offline table, a Model can be configured to customize field selection and data filtering. Models are defined in the project configuration and matched by table name. By default, the plugin uses a generic Postgres model that includes all fields and all features. 

65 

66Base map layers (a map theme or a single layer, as selected in QFieldSync) are rendered through QGIS into a single raster image covering the area of interest at the configured maximum zoom level. The rendered images are cached in the project cache directory for ``mapCacheLifeTime``. 

67 

68Directories listed in the QFieldSync attachment, data and copy settings are added to the package as media files. 

69 

70.. rubric:: Uploading data 

71 

72Data flow QField -> GWS, also called "patching". 

73 

74For each incoming "delta" payload from QField, the plugin extracts the created, updated and deleted features and passes them to the respective Model. The Model is responsible for applying the changes to the Postgres database. 

75 

76A payload is applied all together or not at all: all changes run in one transaction, which is rolled back if any change fails, including changes for unknown layers and updates or deletions of features that do not exist. All models of a QField project must use the same database provider: the configured models and the QGIS layers marked as "offline editable" must refer to the same database connection, otherwise changes from QField are rejected. 

77 

78If a payload fails, the request is answered with status 409. QField then keeps the changes on the device, and the user can push them again once the problem is fixed. 

79 

80Deltas are identified by their content (a hash of the user and the delta), not by the payload id: QField may push the same deltas again under a new payload id, and merges later edits of a feature into a waiting delta. A delta that has already been applied is skipped, a delta that is new, has changed or failed before is applied. 

81 

82The deltas are stored with their status in ``deltas.sqlite`` in the project directory, together with the payload ids they were pushed with, so that QField can poll the status of a payload and show the upload history. They are kept for ``deltaLifeTime``. A delta that stays pending for more than 5 minutes (e.g. after a server crash) counts as failed. 

83 

84.. rubric:: File uploads 

85 

86If a Model supports file uploads, it should contain a virtual file field with ``nameColumn`` and ``contentColumn``:: 

87 

88 actions+ { 

89 type "qfieldcloud" 

90 

91 projects+ { 

92 title "My Project" 

93 provider.path "/path/to/file.qgs" 

94 

95 models+ { 

96 type "postgres" 

97 ... 

98 fields+ { 

99 type "file" 

100 name "virtual_file_field" 

101 contentColumn "file_content" 

102 nameColumn "file_name" 

103 } 

104 } 

105 } 

106 } 

107 

108QField sends uploads in two steps: first, the file path is included along with the feature changes in the delta payload. Later, the actual file content is uploaded in a separate request. The plugin matches the file content to the respective feature based on the ``nameColumn`` value and writes it into ``contentColumn``. If no feature refers to the file, the upload is answered with status 404, and QField drops the file. 

109 

110.. rubric:: File downloads 

111 

112Uploaded files stay on the server unless the file field is listed in ``syncFiles``. A listed field is sent to every user who can read it:: 

113 

114 projects+ { 

115 ... 

116 syncFiles+ { field "virtual_photo" } 

117 syncFiles+ { field "virtual_scan" useBlob true } 

118 } 

119 

120The field name must match exactly one file field in the project models. 

121 

122By default, the content is written into the package as a file, under the path stored in the ``nameColumn``. The QGIS layer needs an "Attachment" widget on the ``nameColumn``. 

123 

124With ``useBlob``, the content is written into the GeoPackage, into a binary column named after the ``contentColumn``. QField sends all attributes of a feature back with every change of the feature, so the blob is transferred again on each edit. QField cannot display binary columns directly; an HTML widget in the QGIS form can show an image as a data URL:: 

125 

126 <img src=expression.evaluate("'data:image/jpeg;base64,' || to_base64(scan_content)")> 

127 

128New files are still taken through an "Attachment" widget on the ``nameColumn`` and uploaded as described above. 

129 

130.. rubric:: Extending 

131 

132Override the packager, patcher and handler classes to customize packaging, patching and request handling. 

133In your custom action class, override ``get_packager()``, ``get_patcher()`` and ``get_handler()`` methods to return your custom classes. 

134Route methods of a custom handler are marked with the ``action_handler.route`` decorator. 

135 

136Example:: 

137 

138 import gws.plugin.qfieldcloud.action 

139 import gws.plugin.qfieldcloud.action_handler 

140 import gws.plugin.qfieldcloud.patcher 

141 

142 class MyPatcher(gws.plugin.qfieldcloud.patcher.Object): 

143 def perform_operations_for_model(self, me, ops): 

144 ... 

145 super().perform_operations_for_model(me, ops) 

146 

147 class MyHandler(gws.plugin.qfieldcloud.action_handler.Handler): 

148 @gws.plugin.qfieldcloud.action_handler.route('GET api/v1/status') 

149 def on_get_status(self): 

150 ... 

151 return super().on_get_status() 

152 

153 class MyAction(gws.plugin.qfieldcloud.action.Object): 

154 def get_patcher(self): 

155 return MyPatcher() 

156 

157 def get_handler(self): 

158 return MyHandler(self) 

159""" 

160 

161from . import ( 

162 action, 

163 packager, 

164 patcher, 

165 caps, 

166) 

167 

168__all__ = [ 

169 'action', 

170 'packager', 

171 'patcher', 

172 'caps', 

173]