en-US/about_AU.help.txt

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
# Chocolatey Automatic Package Updater Module
 
This PowerShell module implements functions that can be used to automate [Chocolatey](https://chocolatey.org) package updates.
 
To learn more about Chocolatey automatic packages, please refer to the relevant [documentation](https://github.com/chocolatey/choco/wiki/AutomaticPackages).
 
## Features
 
- Use only PowerShell to create automatic update script for given package.
- Automatically downloads installers and provides/verifies checksums for x32 and x64 versions.
- Verifies URLs, nuspec versions, remote repository existence etc.
- Can use global variables to change functionality.
- Sugar functions for Chocolatey package maintainers.
- Update single package or any subset of previously created AU packages with a single command.
- Multithread support when updating multiple packages.
- Plugin system when updating everything, with few integrated plugins to send email notifications, save results to gist and push updated packages to git repository.
 
 
## Installation
 
Use one of the following methods:
- PowerShell 5+: [`Install-Module au`](https://www.powershellgallery.com/packages/AU).
- [Download](https://github.com/majkinetor/au/releases/latest) latest 7z package or latest build [artifact](https://ci.appveyor.com/project/majkinetor/au/build/artifacts).
 
AU module requires minimally PowerShell version 4: `$host.Version -ge '4.0'`
 
To quickly start using AU, fork [au-packages-template](https://github.com/majkinetor/au-packages-template) repository and rename it to `au-packages`.
 
**NOTE**: All module functions work from within specific root folder. The folder contains all of your Chocolatey packages.
 
## Creating the package updater script
 
The AU uses `update.ps1` script that package maintainers should create in the package directory. No templates are used, just plain PowerShell.
 
To write the package update script, it is generally required to implement 2 functions: `au_GetLatest` and `au_SearchReplace`.
 
### `au_GetLatest`
 
This function is used to get the latest package information.
 
As an example, the following function uses [Invoke-WebRequest](https://technet.microsoft.com/en-us/library/hh849901.aspx?f=255&MSPPError=-2147217396) to download a page (#1). After that it takes a `href` attribute from the first page link that ends with `.exe` word as a latest URL for the package (#2). Then it conveniently splits the URL to get the latest version for the package (#3), a step that is highly specific to the URL but very easy to determine.
 
```powershell
function global:au_GetLatest {
     $download_page = Invoke-WebRequest -Uri $releases #1
     $regex = '.exe$'
     $url = $download_page.links | ? href -match $regex | select -First 1 -expand href #2
     $version = $url -split '-|.exe' | select -Last 1 -Skip 2 #3
     return @{ Version = $version; URL32 = $url }
}
```
 
The returned version is later compared to the one in the nuspec file and if remote version is higher, the files will be updated. The returned keys of this HashTable are available via global variable `$global:Latest` (along with some keys that AU generates). You can put whatever data you need in the returned HashTable - this data can be used later in `au_SearchReplace`.
 
 
### `au_SearchReplace`
 
Function returns HashTable containing search and replace data for any package file in the form:
 
```powershell
    @{
        file_path1 = @{
            search1 = replace1
            ...
            searchN = replaceN
        }
        file_path2 = @{ ... }
        ...
    }
```
 
Search and replace strings are operands for PowerShell [replace](http://www.regular-expressions.info/powershell.html) operator. You do not have to write them most of the time however, they are rarely changed.
 
File paths are relative to the package directory. The function can use `$global:Latest` variable to get any type of information obtained when `au_GetLatest` was executed along with some AU generated data such as `PackageName`, `NuspecVersion` etc.
 
The following example illustrates the usage:
 
```powershell
function global:au_SearchReplace {
    @{
        "tools\chocolateyInstall.ps1" = @{
            "(^[$]url32\s*=\s*)('.*')" = "`$1'$($Latest.URL32)'" #1
            "(^[$]checksum32\s*=\s*)('.*')" = "`$1'$($Latest.Checksum32)'" #2
        }
    }
}
```
 
Here, line of the format `$url32 = '<package_url>'` in the file `tools\chocolateyInstall.ps1` will have its quoted string replaced with latest URL (#1). The next line replaces value of the variable `$checksum32` on the start of the line with the latest checksum that is automatically injected in the `$Latest` variable by the AU framework (#2). Replacement of the latest version in the nuspec file is handled automatically.
 
**NOTE**: The search and replace works on lines, multiple lines can not be matched with single regular expression.
 
### Update
 
With above functions implemented calling the `Update-Package` (alias `update`) function from the AU module will update the package when needed.
 
You can then update the individual package by running the appropriate `update.ps1` script from within the package directory:
 
```
PS C:\chocolatey\dngrep> .\update.ps1
dngrep - checking updates using au version 2016.9.14
nuspec version: 2.8.15.0
remote version: 2.8.16.0
New version found
Automatic checksum started
Downloading dngrep 32 bit
  from 'https://github.com/dnGrep/dnGrep/releases/download/v2.8.16.0/dnGREP.2.8.16.x86.msi'
 
Download of dnGREP.2.8.16.x86.msi (3.36 MB) completed.
Package downloaded and hash calculated for 32 bit version
Downloading dngrep 64 bit
  from 'https://github.com/dnGrep/dnGrep/releases/download/v2.8.16.0/dnGREP.2.8.16.x64.msi'
 
Download of dnGREP.2.8.16.x64.msi (3.39 MB) completed.
Package downloaded and hash calculated for 64 bit version
Updating files
  dngrep.nuspec
    updating version: 2.8.15.0 -> 2.8.16.0
  tools\chocolateyInstall.ps1
    (^[$]url32\s*=\s*)('.*') = $1'https://github.com/dnGrep/dnGrep/releases/download/v2.8.16.0/dnGREP.2.8.16.x86.msi'
    (^[$]url64\s*=\s*)('.*') = $1'https://github.com/dnGrep/dnGrep/releases/download/v2.8.16.0/dnGREP.2.8.16.x64.msi'
    (^[$]checksum32\s*=\s*)('.*') = $1'CE4753735148E1F48FE0E1CD9AA4DFD019082F4F43C38C4FF4157F08D346700C'
    (^[$]checksumType32\s*=\s*)('.*') = $1'sha256'
    (^[$]checksum64\s*=\s*)('.*') = $1'025BD4101826932E954AACD3FE6AEE9927A7198FEEFFB24F82FBE5D578502D18'
    (^[$]checksumType64\s*=\s*)('.*') = $1'sha256'
Attempting to build package from 'dngrep.nuspec'.
Successfully created package 'dngrep.2.8.16.0.nupkg'
Package updated
```
 
This is best understood via the example - take a look at the real life package [installer script](https://github.com/majkinetor/au-packages/blob/master/dngrep/tools/chocolateyInstall.ps1) and its [AU updater](https://github.com/majkinetor/au-packages/blob/master/dngrep/update.ps1).
 
### Checks
 
The `update` function does the following checks:
 
- The `$Latest.Version` will be checked to match a valid nuspec pattern.
- Any hash key that starts with the word `Url`, will be checked for existence and MIME textual type, since binary is expected here.
- If the remote version is higher then the nuspec version, the Chocolatey site will be checked for existence of this package version (this works for unpublished packages too). This allows multiple users to update same set of packages without a conflict. Besides, this feature makes it possible not to persist state between the updates as once the package is updated and pushed, the next update will not push the package again. To persist the state of updated packages you can use for instance [Git](https://github.com/majkinetor/au/blob/master/AU/Plugins/Git.ps1) plugin which saves the updated and published packages to the git repository.
- The regex patterns in `au_SearchReplace` will be checked for existence.
 
If any of the checks fails, package will not get updated. This feature releases you from the worries about how precise is your pattern match in both `au_` functions - if for example, a vendor site changes, the package update will fail because of the wrongly parsed data.
 
For some packages, you may want to disable some of the checks by specifying additional parameters of the `update` function (not all can be disabled):
 
| Parameter | Description |
| --------- | ------------ |
| `NoCheckUrl` | Disable URL checks |
| `NoCheckChocoVersion` | Disable the Chocolatey site check |
| `ChecksumFor none` | Disable automatic checksum |
 
### Automatic checksums
 
When new version is available, the `update` function will by default download both x32 and x64 versions of the installer and calculate the desired checksum. It will inject this info in the `$global:Latest` HashTable variable so you can use it via `au_SearchReplace` function to update hashes. The parameter `ChecksumFor` can contain words `all`, `none`, `32` or `64` to further control the behavior.
 
You can disable this feature by calling update like this:
 
    update -ChecksumFor none
 
You can define the hash algorithm by returning corresponding `ChecksumTypeXX` hash keys in the `au_GetLatest` function:
 
    return @{ ... ChecksumType32 = 'sha512'; ... }
 
If the checksum is actually obtained from the vendor's site, you can provide it along with its type (SHA256 by default) by returning corresponding `ChecksumXX` hash keys in the `au_GetLatest` function:
 
    return @{ ... Checksum32 = 'xxxxxxxx'; ChecksumType32 = 'md5'; ... }
 
If the `ChecksumXX` hash key is present, the AU will change to checksum verification mode - it will download the installer and verify that its checksum matches the one provided. If the key is not present, the AU will calculate hash with the given `ChecksumTypeXX` algorithm.
 
**NOTE**: This feature works by monkey patching the `Get-ChocolateyWebFile` helper function and invoking the `chocolateyInstall.ps1` afterwards for the package in question. This means that it downloads the files using whatever method is specified in the package installation script.
 
### Force update
 
You can force the update even if no new version is found by using the parameter `Force` (or global variable `$au_Force`). This can be useful for testing the update and bug fixing, recalculating the checksum after the package was created and already pushed to Chocolatey or if URLs to installer changed without change in version.
 
**Example**:
 
```
PS C:\chocolatey\cpu-z.install> $au_Force = $true; .\update.ps1
cpu-z.install - checking updates
nuspec version: 1.77
remote version: 1.77
No new version found, but update is forced
Automatic checksum started
...
Updating files
  cpu-z.install.nuspec
    updating version using Chocolatey fix notation: 1.77 -> 1.77.0.20160814
...
```
 
Force option changes how package version is used. Without force, the `NuspecVersion` determines what is going on. Normally, if `NuspecVersion` is lower or equal then the `RemoteVersion` update happens. With `Force` this changes:
 
1. If `NuspecVersion` is lower then `RemoteVersion`, Force is ignored and update happens as it would normally
2. If `NuspecVersion` is the same as the `RemoteVersion`, the version will change to chocolatey fix notation.
3. If the `NuspecVersion` is already using chocolatey fix notation, the version will be updated to fix notation for the current date.
4. If the `NuspecVersion` is higher then the `RemoteVersion` update will happen but `RemoteVersion` will be used.
 
Points 2-4 do not apply if you set the explicit version using the variable `au_Version`.
 
[Chocolatey fix notation](https://github.com/chocolatey/choco/wiki/CreatePackages#package-fix-version-notation) changes a version so that current date is added in the _revision_ component of the package version in the format `yyyyMMdd`. More precisely:
 
- chocolatey _fix version_ always ends up in to the _Revision_ part of the package version;
- existing _fix versions_ are changed to contain the current date;
- if _revision_ part is present in the package version and it is not in the _chocolatey fix notation_ form, AU will keep the existing version but notify about it;
 
### Global variables
 
To avoid changing the `./update.ps1` when troubleshooting or experimenting you can set up any **already unset** `update` parameter via global variable. The names of global variables are the same as the names of parameters with the prefix `au_`. As an example, the following code will change the update behavior so that URL is not checked for existence and MIME type and update is forced:
 
    $au_NoCheckUrl = $au_Force = $true
    ./update.ps1
 
This is the same as if you added the parameters to `update` function inside the `./update.ps1` script:
 
    update -NoCheckUrl -Force
 
however, its way easier to setup global variable with manual intervention on multiple packages.
 
### Reusing the AU updater with metapackages
 
Metapackages can reuse an AU updater of its dependency by the following way:
 
- In the dependent updater, instead of calling the `update` directly, use construct:
 
  ```
    if ($MyInvocation.InvocationName -ne '.') { update ... }
  ```
 
- In the metapackage updater dot source the dependent updater and override `au_SearchReplace`.
 
This is best understood via example - take a look at the [cpu-z](https://github.com/majkinetor/au-packages/blob/master/cpu-z/update.ps1) AU updater which uses the updater from the [cpu-z.install](https://github.com/majkinetor/au-packages/blob/master/cpu-z.install/update.ps1) package on which it depends. It overrides the `au_SearchReplace` function and the `update` call but keeps the `au_GetLatest`.
 
## Updating all packages
 
You can update all packages and optionally push them to the Chocolatey repository with a single command. Function `Update-AUPackages` (alias `updateall`) will iterate over `update.ps1` scripts and execute each in a separate thread. If it detects that a package is updated it will optionally try to push it to the Chocolatey repository and may also run configured plugins.
 
For the push to work, specify your Choocolatey API key in the file `api_key` in the script's directory (or its parent directory) or set the environment variable `$Env:api_key`. If none provided cached nuget key will be used.
 
The function will search for packages in the current directory. To override that, use global variable `$au_Root`:
 
    PS> $au_root = 'c:\chocolatey_packages`
    PS> $Options = @{
        Timeout = 10
        Threads = 15
        Push = $true
    }
    PS> updateall -Options $Options
 
    Updating 6 automatic packages at 2016-09-16 22:03:33
    Push is enabled
       copyq is updated to 2.6.1 and pushed
       dngrep had errors during update
           Can't validate URL 'https://github.com/dnGrep/dnGrep/releases/download/v2.8.16.0/dnGREP.2.8.16.x64.msi'
           Exception calling "GetResponse" with "0" argument(s): "The operation has timed out"
       eac has no updates
       pandoc has no updates
       plantuml has no updates
       yed had errors during update
           Can't validate URL 'https://www.yworks.com'
           Invalid content type: text/html
 
    Finished 6 packages after .32 minutes.
    1 updated and 1 pushed.
    2 errors - 2 update, 0 push.
 
 
Use `updateall` parameter `Name` to specify package names via glob, for instance `updateall [a-d]*` would update only packages which names start with the letter 'a' trough 'd'. Add `Push` among options to push successfully built packages to the chocolatey repository.
 
Take a look at the [real life example](http://tiny.cc/v1u1ey) of the update script. To see all available options for `updateall` type `man updateall -Parameter Options`.
 
### Plugins
 
It is possible to specify a custom user logic in `Options` parameter - every key that is of type `[HashTable]` will be considered plugin with the PowerShell script that is named the same as the key. The following code shows how to use 5 integrated plugins:
 
```powershell
    $Options = [ordered]@{
        Timeout = 100
        Threads = 15
        Push = $true
           
        # Save text report in the local file report.txt
        Report = @{
            Type = 'text'
            Path = "$PSScriptRoot\report.txt"
        }
         
        # Then save this report as a gist using your api key and gist id
        Gist = @{
            ApiKey = $Env:github_api_key
            Id = $Env:github_gist_id
            Path = "$PSScriptRoot\report.txt"
        }
 
        # Persist pushed packages to your repository
        Git = @{
            User = ''
            Password = $Env:github_api_key
        }
         
        # Then save run info which can be loaded with Import-CliXML and inspected
        RunInfo = @{
            Path = "$PSScriptRoot\update_info.xml"
        }
 
        # Finally, send an email to the user if any error occurs and attach previously created run info
        Mail = if ($Env:mail_user) {
                @{
                   To = $Env:mail_user
                   Server = 'smtp.gmail.com'
                   UserName = $Env:mail_user
                   Password = $Env:mail_pass
                   Port = 587
                   EnableSsl = $true
                   Attachment = "$PSScriptRoot\$update_info.xml"
                   UserMessage = 'Save attachment and load it for detailed inspection: <code>$info = Import-CliXCML update_info.xml</code>'
                }
        } else {}
    }
```
 
The plugins above - `Report`, `Gist`,`Git`,`RunInfo` and `Mail` - are executed in the given order (hence the `[ordered]` flag) and AU passes them given options and saves the run results.
 
To add custom plugins, specify additional plugin search path via `$Options.PluginPath`. Plugin is a normal PowerShell script and apart from parameters given in its HashTable the AU will send it one more parameter `$Info` that contains current run info. The name of the script file must be the same as that of the key which value is used to pass the parameters to the plugin. If a key with the value of type `[HashTable]` doesn't point to existing PowerShell script it is not considered to be an AU plugin.
 
To temporary disable plugins use `updateall` option `NoPlugins` or global variable `$au_NoPlugins`.
To temporary exclude the AU package from `updateall` procedure add `_` prefix to the package directory name.
 
You can also execute a custom script via ScriptBlock specified via `BeforeEach` and `AfterEach` options. They will receive 2 parameters - package name and Options HashTable which you can use to pass any other parameter.
 
For more information take a look at the wiki section about [plugins](https://github.com/majkinetor/au/wiki/Plugins).
 
### Make a script
 
Its desirable to put everything in a single script `update_all.ps1` so it can be scheduled and called with the given options. Rename `update_all_default.ps1` and uncomment and set the options you need.
 
To make a local scheduled task, use the following code in the directory where your `update_all.ps1` script is found to install it:
 
    $At = '03:00'
    schtasks /create /tn "Update-AUPackages" /tr "powershell -File '$pwd\update_all.ps1'" /sc daily /st $At
 
Its preferable to run the updater on [AppVeyor](https://github.com/majkinetor/au/wiki/AppVeyor).
 
## Other functions
 
Apart from the functions used in the updating process, there are few suggars for regular maintenance of the package:
 
- Test-Package
Quickly test install and/or uninstall of the package from the current directory with optional parameters. This function can be used to start testing in [chocolatey-test-environment](https://github.com/majkinetor/chocolatey-test-environment) via `Vagrant` parameter.
 
- Push-Package (alias `pp`)
Push the latest package using your API key.
 
- Get-AuPackages (alias `gau` or `lsau`)
Returns the list of the packages which have `update.ps1` script in its directory and which name doesn't start with '_'.