Subversion Repositories DevTools

Rev

Rev 5806 | Rev 6177 | Go to most recent revision | Show entire file | Ignore whitespace | Details | Blame | Last modification | View Log | RSS feed

Rev 5806 Rev 6024
Line 1094... Line 1094...
1094 
 
 1094 
 
1095 
checkRelease - Check Release/SBOM packages against dpkg_archive
 1095 
checkRelease - Check Release/SBOM packages against dpkg_archive
1096 
 
 1096 
 
1097 
=head1 SYNOPSIS
 1097 
=head1 SYNOPSIS
1098 
 
 1098 
 
1099 
  jats escrow [options] [name version]
 1099 
  jats checkRelease [options] [name version]
1100 
 
 1100 
 
1101 
 Options:
 1101 
 Options:
1102 
    -help              - brief help message
 1102 
    -help                   - brief help message
1103 
    -help -help        - Detailed help message
 1103 
    -help -help             - Detailed help message
1104 
    -man               - Full documentation
 1104 
    -man                    - Full documentation
1105 
    -sbomid=xxx        - Specify the SBOM to process
 1105 
    -sbomid=xxx             - Specify the SBOM to process
1106 
    -rtagid=xxx        - Specify the Release to process (Optional)
 1106 
    -rtagid=xxx             - Specify the Release to process (Optional)
1107 
    -rootpackage=xxx   - Specifies a root package. In conjunction with -rtagid.
 1107 
    -rootpackage=pkgName    - Specifies a root package. In conjunction with -rtagid.
1108 
    -ignore=name       - Ignore packages with the specified name
 1108 
    -ignore=name            - Ignore packages with the specified name
1109 
    -extract=fname     - Extract files from a previous run
 1109 
    -extract=fname          - Extract files from a previous run
1110 
    -verbose           - Enable verbose output
 1110 
    -verbose                - Enable verbose output
1111 
    -[no]patch         - Ignore/Include patches. Default:Include
 1111 
    -[no]patch              - Ignore/Include patches. Default:Include
1112 
    -[no]test          - Reduced package scanning for test
 1112 
    -[no]test               - Reduced package scanning for test
1113 
 
 1113 
 
1114 
=head1 OPTIONS
 1114 
=head1 OPTIONS
1115 
 
 1115 
 
1116 
=over 8
 1116 
=over 8
1117 
 
 1117 
 
Line 1142... Line 1142...
1142 
The program will determine packages that are in the Release, but not in the
 1142 
The program will determine packages that are in the Release, but not in the
1143 
SBOM.
 1143 
SBOM.
1144 
 
 1144 
 
1145 
Without an SBOM_ID, this option will limit the processing to the specified
 1145 
Without an SBOM_ID, this option will limit the processing to the specified
1146 
release. Less information is generated. This form of the generation may be
 1146 
release. Less information is generated. This form of the generation may be
1147 
combined with B<-rootpackage=xxx> to further limit the set of packages
 1147 
combined with B<-rootpackage=pkgName> to further limit the set of packages
1148 
processed.
 1148 
processed.
1149 
 
 1149 
 
1150 
=item B<-rootpackage=xxx>
 1150 
=item B<-rootpackage=pkgName>
1151 
 
 1151 
 
1152 
This option can be used in conjunction with B<-rtagid=xxx> to limit the
 1152 
This option can be used in conjunction with B<-rtagid=xxx> to limit the
1153 
extraction to named package and all of its dependent packages. The tool will
 1153 
extraction to named package and all of its dependent packages. The tool will
1154 
determine the required version of the package via the specified release.
 1154 
determine the required version of the package via the specified release.
1155 
 
 1155 
 
Line 1161... Line 1161...
1161 
=item B<-extract=name>
 1161 
=item B<-extract=name>
1162 
 
 1162 
 
1163 
This option will process the 'extract' file created in a previous run of this
 1163 
This option will process the 'extract' file created in a previous run of this
1164 
program and restore missing package-versions as specified in the file.
 1164 
program and restore missing package-versions as specified in the file.
1165 
 
 1165 
 
1166 
The command will then create a log file recording packages that could ne be
 1166 
The command will then create a log file recording packages that could not be
1167 
extracted.
 1167 
extracted.
1168 
 
 1168 
 
1169 
This option does not not interwork with many of the other command line options.
 1169 
This option does not interwork with many of the other command line options.
1170 
This option cannot be used in conjunction with the -rtagid, -sbomid, rootpackage
 1170 
This option cannot be used in conjunction with the -rtagid, -sbomid, rootpackage
1171 
and -nopatch.
 1171 
and -nopatch.
1172 
 
 1172 
 
1173 
=item B<-[no]patch>
 1173 
=item B<-[no]patch>
1174 
 
 1174 
 
Line 1187... Line 1187...
1187 
 
 1187 
 
1188 
=back
 1188 
=back
1189 
 
 1189 
 
1190 
=head1 DESCRIPTION
 1190 
=head1 DESCRIPTION
1191 
 
 1191 
 
1192 
This program is a tool for extracting Escrow build information.
 1192 
This program is a tool for recovering archived package from AWS S3 storage.
1193 
The program has two modes of operation:
 1193 
The program has two modes of operation:
1194 
 
 1194 
 
1195 
=over 8
 1195 
=over 8
1196 
 
 1196 
 
1197 
=item 1
 1197 
=item 1
Line 1199... Line 1199...
1199 
Generation. Generate files describing packages within an SBOM/Release/
 1199 
Generation. Generate files describing packages within an SBOM/Release/
1200 
Package.
 1200 
Package.
1201 
 
 1201 
 
1202 
=item 2
 1202 
=item 2
1203 
 
 1203 
 
1204 
Extraction  Supervise extraction of source trees.
 1204 
Restoration.  Supervise restoration of packages from S3.
1205 
 
 1205 
 
1206 
=back
 1206 
=back
1207 
 
 1207 
 
1208 
=head2 Generation Operations
 1208 
=head2 Operations
1209 
 
 1209 
 
1210 
This program has several modes of operation. The mode is determined from the
 1210 
This program has several modes of operation. The mode is determined from the
1211 
command line arguments provided.
 1211 
command line arguments provided.
1212 
 
 1212 
 
1213 
=over 8
 1213 
=over 8
1214 
 
 1214 
 
1215 
=item   Full Escrow
 1215 
=item   SBOM Restore
1216 
 
 1216 
 
1217 
This mode requires an SBOM_ID. If an RTAG_ID is also provided, then additional
 1217 
This mode requires an SBOM_ID. If an RTAG_ID is also provided, then additional
1218 
information will be generated.
 1218 
information will be generated.
1219 
 
 1219 
 
1220 
=item   Release Escrow
 1220 
=item   Release Restore
1221 
 
 1221 
 
1222 
If only an RTAG_ID is provided then the processing wil be limited to the
 1222 
If only an RTAG_ID is provided then the processing will be limited to the
1223 
packages involved in creating the specified release.
 1223 
packages involved in creating the specified release.
1224 
 
 1224 
 
1225 
If a 'rootpackage' name is provided, then the processing is limited to
 1225 
If a 'rootpackage' name is provided, then the processing is limited to
1226 
packages that depend on the named package.
 1226 
packages that depend on the named package.
1227 
 
 1227 
 
1228 
=item   Single Package
 1228 
=item   Single Package Restore
1229 
 
 1229 
 
1230 
If a package name and a package version are specified on the command line,
 1230 
If a package name and a package version are specified on the command line,
1231 
then the processing will be limited to the specified package and ist dependents.
 1231 
then the processing will be limited to the specified package and it's dependents.
1232 
No release related information will be provided.
 1232 
No release related information will be provided.
1233 
 
 1233 
 
1234 
=back
 1234 
=back
1235 
 
 1235 
 
1236 
The 'Full Escrow' extract is the complete operation. All others are sub-sets of
 1236 
The 'SBOM Restore' extract is the complete operation. All others are sub-sets of
1237 
this processing. The complete processing is:
 1237 
this processing. The complete processing is:
1238 
 
 1238 
 
1239 
=over 8
 1239 
=over 8
1240 
 
 1240 
 
1241 
=item *
 1241 
=item *
Line 1254... Line 1254...
1254 
 
 1254 
 
1255 
Determine all the dependent packages for all packages encountered
 1255 
Determine all the dependent packages for all packages encountered
1256 
 
 1256 
 
1257 
=item *
 1257 
=item *
1258 
 
 1258 
 
1259 
Generate a list of jats commands to extract the package source
 1259 
Generate a list of package-versions that are not present in dpkg_archive.
1260 
 
 - 
 
- 
 
 1260 
These are 'missing'. This tools can be used in 'Restore' mode to restore these
1261 
=item *
 1261 
packages.
1262 
 
 1262 
 
1263 
Generate a file describing the build order
 1263 
=back
1264 
 
 1264 
 
1265 
=item *
 - 
 
- 
 
 1265 
This may take some time, as a typical SBOM may contain many hundreds of packages.
1266 
 
 1266 
 
1267 
Generate a file describing the packages that cannot be built
 1267 
The program will display a list of files that have been created.
1268 
 
 1268 
 
1269 
=item *
 1269 
=head2 Restoration Operations
1270 
 
 1270 
 
1271 
Generate an HTML file with extensive cross reference information
 1271 
Given an 'missing' file from a previous run of this program the program will:
1272 
 
 1272 
 
1273 
=over 8
 1273 
=over 8
1274 
 
 1274 
 
1275 
=item *
 1275 
=item *
1276 
 
 1276 
 
1277 
List of all packages with references into Release Manager
 - 
 
1278 
 
 - 
 
1279 
=item *
 - 
 
1280 
 
 - 
 
1281 
List of all packages showing dependent packages
 - 
 
1282 
 
 - 
 
1283 
=item *
 - 
 
1284 
 
 - 
 
1285 
List of all packages showing consumer packages
 - 
 
1286 
 
 - 
 
1287 
=item *
 - 
 
1288 
 
 - 
 
1289 
List of all packages for which multiple versions are required
 - 
 
1290 
 
 - 
 
1291 
=item *
 - 
 
1292 
 
 - 
 
1293 
Details of packages that are not built.
 1277 
Parse the 'missing' file
1294 
 
 1278 
 
1295 
=item *
 1279 
=item *
1296 
 
 1280 
 
1297 
Build order
 1281 
Copy the required package versions from AWS S3 and restore them into dpkg_archive.
1298 
 
 1282 
 
1299 
=item *
 - 
 
1300 
 
 - 
 
- 
 
 1283 
If the S3 copies have been transferred to 'GLACIER", then a message will be displayed. The user will need to instigate
1301 
Build machines and built types
 1284 
the recovery from Glacier using traditional AWS tools. Currently this process is not automated by this tool.
1302 
 
 1285 
 
1303 
=item *
 1286 
=item *
1304 
 
 1287 
 
1305 
Deployed target nodes, with references into Deployment Manager
 1288 
Create a log file showing packages that could not be restored.
1306 
 
 1289 
 
1307 
=back
 1290 
=back
1308 
 
 1291 
 
1309 
=back
 - 
 
- 
 
 1292 
The 'Restoration' operation can only be performed on the package server machine by the 'buildadm' user.
- 
 
 1293 
The operation is not available to other users.
1310 
 
 1294 
 
1311 
This may take some time, as a typical escrow build may contain many hundreds of packages.
 1295 
=head2 Format of missing file
1312 
 
 1296 
 
1313 
The program will display a list of files that have been created.
 1297 
The 'missing' file is a file generated by this script in the 'Generation' mode and consumed by this script in the 'Restoration' mode.
1314 
 
 1298 
 
1315 
=head2 Extraction Operations
 - 
 
1316 
 
 - 
 
1317 
Given an 'extract' file from a previous run of this program the program will:
 1299 
The file can be manually created if required. The file has a simple format of:
1318 
 
 1300 
 
1319 
=over 8
 1301 
=over 8
1320 
 
 1302 
 
1321 
=item *
 1303 
=item * One item per line
1322 
 
 1304 
 
1323 
Parse the 'extract' file
 1305 
=item * Both Unix and Windows line terminators are supported
1324 
 
 1306 
 
1325 
=item *
 1307 
=item * There is no support for comments. All lines are processed
1326 
 
 1308 
 
1327 
Create subdirectories for each package version within the file. This is done
 1309 
=item * Each line consists of the package name and the package version joined with a '/'.
1328 
in such a way that no views are left in place.
 - 
 
1329 
 
 1310 
 
1330 
=item *
 1311 
=back
1331 
 
 1312 
 
1332 
Create a log file showing packages that could not be extracted.
 1313 
=head2 Example of missing file
1333 
 
 1314 
 
- 
 
 1315 
A simple 'missing' file with two entries.
1334 
=back
 1316 
 
- 
 
 1317 
VIXubuntu1404LTS_VMcfg/1.0.0000.tool
- 
 
 1318 
VIXubuntu1402LTS_VMcfg/1.0.0000.tool
1335 
 
 1319 
 
1336 
=cut
 1320 
=cut
1337 
 
 1321