Modules/businessdev.ALbuild.Core/Public/Clear-ALbuildCache.ps1
|
function Clear-ALbuildCache { <# .SYNOPSIS Prunes old, unused ALbuild cache content to reclaim disk space on a build agent. .DESCRIPTION Build agents accumulate large caches that ALbuild never trims on its own: the extracted artifact/symbol cache (ArtifactCacheFolder), the dependency package cache (PackageCacheFolder) and the Business Central artifact cache (BcArtifactCacheFolder) - the last two of which grow a new sub-folder per BC version and per package, unbounded. This cmdlet removes cache entries that have not been written to within a retention window, keeping recent ones. The cache locations are read from Get-ALbuildConfig, so it prunes exactly the folders the tasks use (including any relocated to a roomier drive). Each cache's immediate child entries are the pruning unit; an entry is removed when its last-write age exceeds -KeepDays, except the newest -KeepLatest entries per cache are always retained as a safety floor. Age is measured by LastWriteTime (last-access time is unreliable - Windows disables it by default). Supports -WhatIf/-Confirm (it is destructive), reports the space reclaimed per cache, and logs every removal so a scheduled cleanup is auditable from the pipeline log. With -IncludeDocker it also prunes unused Docker images and stopped containers older than the same window. .PARAMETER KeepDays Retention window in days. Cache entries not written within this many days are removed. Default 30. .PARAMETER KeepLatest Always keep at least this many most-recently-written entries per cache, regardless of age (a safety floor so a cache is never emptied). Default 0. .PARAMETER Cache Which caches to prune: All (default), Artifacts, Packages or BcArtifacts. .PARAMETER IncludeDocker Also run 'docker image prune' (unused images) and 'docker container prune' (stopped containers) filtered to the same -KeepDays window. Skipped with a note if the docker CLI is unavailable. Images carrying any -ProtectLabel are excluded - see that parameter. .PARAMETER ProtectLabel Docker image labels that must NEVER be pruned by -IncludeDocker. Default 'albuild.image', which is the label New-BcImage stamps on the version-specific BC image cache. Why this is not optional: 'docker image prune -a' removes ALL unused images, not just dangling ones, and New-BcContainer invokes this cmdlet automatically when free disk drops below its floor. Without the exclusion, the one cmdlet that runs when the disk is tight would delete the image cache - which is largest, and most valuable, at exactly that moment. Each deleted image then costs a multi-minute rebuild. Retention of the image cache belongs to Optimize-BcImageCache, which evicts by least-recent-use against a budget and never touches an image a run has pinned. Pass an empty array to opt out (for a deliberate full reclaim). .EXAMPLE Clear-ALbuildCache -WhatIf Show what a default 30-day prune would remove, without deleting anything. .EXAMPLE Clear-ALbuildCache -KeepDays 14 -KeepLatest 2 -IncludeDocker Keep 14 days (but never fewer than the 2 newest per cache) and also prune old Docker content. .OUTPUTS PSCustomObject: per-cache Removed count / FreedBytes / FreedText plus a TotalFreedBytes/Text. #> [CmdletBinding(SupportsShouldProcess, ConfirmImpact = 'High')] [OutputType([PSCustomObject])] param( [ValidateRange(0, [int]::MaxValue)] [int] $KeepDays = 30, [ValidateRange(0, [int]::MaxValue)] [int] $KeepLatest = 0, [ValidateSet('All', 'Artifacts', 'Packages', 'BcArtifacts')] [string] $Cache = 'All', [switch] $IncludeDocker, [string[]] $ProtectLabel = @('albuild.image') ) $config = Get-ALbuildConfig # Depth = how deep below the cache root a single prunable ENTRY sits. # # The artifact caches are laid out <root>\<type>\<version>\<country|platform> - so the unit that can # be reclaimed independently is the VERSION folder, at depth 2. Pruning at depth 1 (the historical # behaviour) had two failure modes: it could only ever delete a whole <type> tree - every BC version # on the agent at once - and it never fired in practice, because <type>'s LastWriteTime is refreshed # whenever any new version lands underneath it. The retention this cmdlet documents ("a new sub-folder # per BC version") only works per version. # # The package cache keeps one folder per package directly under the root, so it stays at depth 1. $targets = [ordered]@{} if ($Cache -in @('All', 'Artifacts')) { $targets['Artifacts'] = @{ Path = $config.ArtifactCacheFolder; Depth = 2 } } if ($Cache -in @('All', 'Packages')) { $targets['Packages'] = @{ Path = $config.PackageCacheFolder; Depth = 1 } } if ($Cache -in @('All', 'BcArtifacts')) { $targets['BcArtifacts'] = @{ Path = $config.BcArtifactCacheFolder; Depth = 2 } } $cutoff = (Get-Date).AddDays(-$KeepDays) Write-ALbuildLog "Pruning ALbuild caches: keep entries written on/after $($cutoff.ToString('yyyy-MM-dd')) (KeepDays=$KeepDays), always keep the newest $KeepLatest per cache." $humanize = { param([long] $Bytes) $units = 'B', 'KB', 'MB', 'GB', 'TB'; $i = 0; $n = [double]$Bytes while ($n -ge 1024 -and $i -lt $units.Count - 1) { $n /= 1024; $i++ } '{0:0.##} {1}' -f $n, $units[$i] } $folderSize = { param([string] $Path) try { [long](Get-ChildItem -LiteralPath $Path -Recurse -File -Force -ErrorAction SilentlyContinue | Measure-Object -Property Length -Sum).Sum } catch { [long]0 } } # Acquire the cache locks covering an entry: the entry itself plus every immediate child, because # Get-BcArtifact locks the leaf it extracts into (<version>\w1, <version>\platform), not the version. # All-or-nothing - a partial set is released immediately so we can never hold half a lock set. $acquireLocks = { param([string] $EntryPath) $paths = @($EntryPath) + @( Get-ChildItem -LiteralPath $EntryPath -Directory -Force -ErrorAction SilentlyContinue | ForEach-Object { $_.FullName } ) $held = [System.Collections.Generic.List[object]]::new() foreach ($p in $paths) { $mutex = New-Object System.Threading.Mutex($false, (Get-ALbuildCacheLockName -Path $p)) $got = $false try { $got = $mutex.WaitOne([TimeSpan]::FromSeconds(5)) } catch [System.Threading.AbandonedMutexException] { $got = $true } # prior holder crashed; we own it if (-not $got) { $mutex.Dispose() foreach ($h in $held) { $h.ReleaseMutex(); $h.Dispose() } return [PSCustomObject]@{ Acquired = $false; BusyPath = $p; Mutexes = @() } } $held.Add($mutex) } return [PSCustomObject]@{ Acquired = $true; BusyPath = $null; Mutexes = $held.ToArray() } } $releaseLocks = { param([object] $Locks) foreach ($m in @($Locks.Mutexes)) { try { $m.ReleaseMutex() } catch { } $m.Dispose() } } $results = [System.Collections.Generic.List[object]]::new() $totalFreed = [long]0 foreach ($name in $targets.Keys) { $root = $targets[$name].Path $depth = $targets[$name].Depth if ([string]::IsNullOrWhiteSpace($root) -or -not (Test-Path -LiteralPath $root)) { Write-ALbuildLog -Level Verbose " [$name] '$root' does not exist; nothing to prune." $results.Add([PSCustomObject]@{ Cache = $name; Path = $root; Removed = 0; FreedBytes = [long]0; FreedText = '0 B' }) continue } # Walk down to the entry level. Intermediate levels are containers, never prune units. $entryFolders = @([PSCustomObject]@{ FullName = $root }) for ($level = 1; $level -le $depth; $level++) { $entryFolders = @($entryFolders | ForEach-Object { Get-ChildItem -LiteralPath $_.FullName -Directory -Force -ErrorAction SilentlyContinue }) if ($entryFolders.Count -eq 0) { break } } $entries = @($entryFolders | Sort-Object LastWriteTime -Descending) # Safety floor: never consider the newest -KeepLatest entries for removal. $candidates = if ($KeepLatest -gt 0 -and $entries.Count -gt $KeepLatest) { $entries[$KeepLatest..($entries.Count - 1)] } elseif ($KeepLatest -gt 0) { @() } else { $entries } $stale = @($candidates | Where-Object { $_.LastWriteTime -lt $cutoff }) $removed = 0; $freed = [long]0 foreach ($dir in $stale) { $size = & $folderSize $dir.FullName if ($PSCmdlet.ShouldProcess($dir.FullName, "Remove cached entry (last modified $($dir.LastWriteTime.ToString('yyyy-MM-dd')), $(& $humanize $size))")) { # Lock before removing. Get-BcArtifact holds the lock of the LEAF folder it extracts into # (<version>\w1, <version>\platform), so a version entry is only safe to delete once every # leaf underneath it is free. Acquiring with a short timeout - and skipping the entry when # that fails - is deliberate: a busy cache entry is one another build is actively using, # and disk pressure is never a good enough reason to pull files out from under it. $locks = & $acquireLocks $dir.FullName if (-not $locks.Acquired) { Write-ALbuildLog -Level Warning " [$name] skipped '$($dir.Name)': in use by another build (lock held on '$($locks.BusyPath)')." continue } try { Remove-Item -LiteralPath $dir.FullName -Recurse -Force -ErrorAction Stop $removed++; $freed += $size Write-ALbuildLog " [$name] removed '$($dir.Name)' (modified $($dir.LastWriteTime.ToString('yyyy-MM-dd')), freed $(& $humanize $size))." } catch { Write-ALbuildLog -Level Warning " [$name] could not remove '$($dir.FullName)': $($_.Exception.Message)" } finally { & $releaseLocks $locks } } else { $freed += $size } # -WhatIf: still report what would be freed } $totalFreed += $freed $entryWord = if ($stale.Count -eq 1) { 'entry' } else { 'entries' } Write-ALbuildLog -Level Success " [$name] $($stale.Count) stale $entryWord of $($entries.Count); freed $(& $humanize $freed)." $results.Add([PSCustomObject]@{ Cache = $name; Path = $root; Removed = $removed; FreedBytes = $freed; FreedText = (& $humanize $freed) }) } if ($IncludeDocker) { if (Get-Command -Name docker -ErrorAction SilentlyContinue) { $until = "$([int]($KeepDays * 24))h" # 'image prune -a' removes every unused image, so the BC image cache has to be excluded by # label or this prune destroys it - and New-BcContainer triggers this cmdlet automatically # under disk pressure, i.e. exactly when that cache is biggest. Docker AND-combines filter # keys and honours the negated form (verified against engine 28.5.1). $imageArgs = Get-ALbuildDockerPruneArgument -KeepDays $KeepDays -ProtectLabel $ProtectLabel if ($PSCmdlet.ShouldProcess("docker (until=$until)", 'Prune unused images and stopped containers')) { $protectNote = if (@($ProtectLabel).Count) { " (keeping images labelled $($ProtectLabel -join ', '))" } else { '' } Write-ALbuildLog " [Docker] pruning stopped containers and unused images older than $until$protectNote..." try { (& docker container prune -f --filter "until=$until" 2>&1) | ForEach-Object { Write-ALbuildLog -Level Verbose " $_" } (& docker @imageArgs 2>&1) | ForEach-Object { Write-ALbuildLog " $_" } } catch { Write-ALbuildLog -Level Warning " [Docker] prune failed: $($_.Exception.Message)" } } } else { Write-ALbuildLog -Level Warning ' [Docker] -IncludeDocker requested but the docker CLI was not found; skipping.' } } Write-ALbuildLog -Level Success "ALbuild cache prune complete: reclaimed $(& $humanize $totalFreed) across $($results.Count) cache(s)." [PSCustomObject]@{ Caches = $results.ToArray() TotalFreedBytes = $totalFreed TotalFreedText = (& $humanize $totalFreed) KeepDays = $KeepDays KeepLatest = $KeepLatest } } |