Modules/businessdev.ALbuild.Containers/Public/Get-BcArtifactSymbolFolder.ps1
|
function Get-BcArtifactSymbolFolder { <# .SYNOPSIS Returns the symbol (package cache) folders a host AL compile needs from a downloaded BC artifact. .DESCRIPTION Collects every folder of a Get-BcArtifact result that provides compiled first-party symbols for a host (AL Tool) compile, in one place instead of each caller re-deriving the layout: * '<application>/Extensions' - first-party app symbols shipped with the country artifact. * '<application>/Applications.<COUNTRY>' - the compiled country apps INCLUDING the test toolkit (Tests-TestLibraries, Library Assert, Test Runner, ...). Localized artifacts only. * the platform folder holding 'System.app' - the AL system/runtime symbols. The W1 artifact ships no 'Applications.W1' folder - its compiled test toolkit lives scattered through the PLATFORM artifact instead ('Applications/TestFramework/**', 'Applications/BaseApp/Test', 'Applications/System Application/Test', ...). Without it, compiling a test app against W1 fails with AL1022 ('Tests-TestLibraries ... could not be found'). So when the country artifact has no 'Applications.*' folder, the platform's apps are staged once into '.albsym-<country>' beside the platform artifact and that folder is returned too (keyed to the artifact version, so it is reused across builds). Staging is serialised on the artifact cache lock and published by an atomic rename, so a concurrent build never sees a half-copied symbol folder: * W1 sandbox (an 'Extensions' folder IS present): only the test toolkit is missing - stage just those apps (paths matching 'TestFramework', a 'Test' segment, or a 'Test Library' name). * On-premises (NO 'Extensions' and NO 'Applications.<country>'): the platform 'Applications' tree is the only source of first-party symbols, so stage them ALL (the business apps Application / Base Application / Business Foundation under '<App>/Source', plus the toolkit), or the host compile fails with AL1022 for 'Microsoft Application' / 'Base Application'. .PARAMETER Artifact The artifact object returned by Get-BcArtifact (ApplicationPath / PlatformPath). .PARAMETER Force Re-stage the W1 toolkit symbols even when the staging folder already exists. .EXAMPLE $artifact = Get-BcArtifact -ArtifactUrl $url $symbols = Get-BcArtifactSymbolFolder -Artifact $artifact Invoke-BcCompiler -ProjectFolder .\app -PackageCachePath (@('.alpackages') + $symbols) .OUTPUTS System.String[] - existing folders, ready for the compiler's package cache path. #> [CmdletBinding()] [OutputType([string[]])] param( [Parameter(Mandatory)] [ValidateNotNull()] [object] $Artifact, [switch] $Force ) $paths = [System.Collections.Generic.List[string]]::new() $extensions = Join-Path $Artifact.ApplicationPath 'Extensions' $hasExtensions = Test-Path -LiteralPath $extensions if ($hasExtensions) { $paths.Add($extensions) } # Compiled, country-specific first-party apps incl. the test toolkit (localized artifacts only). $countryApps = @(Get-ChildItem -LiteralPath $Artifact.ApplicationPath -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -match '^Applications\..+' }) foreach ($folder in $countryApps) { $paths.Add($folder.FullName) } if ($Artifact.PlatformPath) { # The platform 'System.app' (AL system/runtime symbols) sits in a version-specific folder. $systemApp = Get-ChildItem -LiteralPath $Artifact.PlatformPath -Recurse -Filter 'System.app' -File -ErrorAction SilentlyContinue | Select-Object -First 1 if ($systemApp) { $paths.Add($systemApp.Directory.FullName) } # No 'Applications.<country>' folder: the compiled first-party apps the compiler needs live in an # 'Applications' tree instead - stage them once beside the platform artifact. # * W1 sandbox: the country artifact's 'Extensions' folder already provides the business apps # (Application, Base Application, ...); only the platform's test toolkit is missing, so stage # just that (TestFramework / a 'Test' segment / a 'Test Library' name). # * On-premises: there is no 'Extensions' (nor 'Applications.<country>') folder at all. The # *application* artifact carries an 'Applications' tree with the full country-specific # first-party set INCLUDING localization (e.g. the German 'Delivery Reminder' objects in the # DE Base Application) - the platform's W1 'Applications' tree does NOT. So prefer the # application artifact's tree and stage every app, or the compile fails with AL1022 for # 'Microsoft Application' / 'Base Application' or AL0247/AL0185 for localized objects. if ($countryApps.Count -eq 0) { $appApps = Join-Path $Artifact.ApplicationPath 'Applications' $platformApps = Join-Path $Artifact.PlatformPath 'Applications' # Choose the source by artifact kind, keyed on 'Extensions' (present only on sandbox): # * Sandbox (has 'Extensions'): the business apps come from 'Extensions'; only the test # toolkit is missing and it lives in the PLATFORM 'Applications' tree - never in the # application artifact. Use the platform tree (whether the country is W1 or localized). # * On-prem (no 'Extensions'): the full country first-party set INCLUDING localization is # in the APPLICATION artifact's 'Applications' tree; use it (fall back to the platform # tree only if the application artifact somehow has none). $sourceApps = if (-not $hasExtensions -and (Test-Path -LiteralPath $appApps)) { $appApps } elseif (Test-Path -LiteralPath $platformApps) { $platformApps } else { '' } if ($sourceApps) { # What the compiler must end up with, decided BEFORE looking at the cache: the file list # is what tells a complete staging from a half-copied one, and it is the only honest # description of what a cached folder was supposed to contain. $sourceAppFiles = @(Get-ChildItem -LiteralPath $sourceApps -Recurse -Filter '*.app' -File -ErrorAction SilentlyContinue) if ($hasExtensions) { $sourceAppFiles = @($sourceAppFiles | Where-Object { $_.FullName -match '(?i)[\\/]TestFramework[\\/]|[\\/]Test[\\/]|Test Library' }) } # KEYED BY COUNTRY. The staging folder sits beside the PLATFORM artifact, and the platform # is shared by every country of a version - but for on-premises the staged set comes from # the APPLICATION artifact and is country-specific (the German Base Application carries # objects the W1 one does not). One folder for both countries means the second build to # arrive either overwrites the first one's symbols or compiles against them. # # Run 27372 hit both halves of that on 27.5.46862.0: one worker compiled against an # incomplete set - AL1022 for 'Base Application', which then blocked four dependent # products - while the other's publish lost the race and was swallowed as a warning. $country = "$($Artifact.Country)".ToLowerInvariant() if (-not $country) { $country = 'unknown' } # Short name on purpose: BC artifacts nest close to the 260-char MAX_PATH limit. $staging = Join-Path $Artifact.PlatformPath ".albsym-$country" # A cached folder is usable only when it holds exactly what THIS resolution asked for. # The marker records the filter ('toolkit' for a W1 sandbox vs 'full' for on-prem), the # source tree (the country 'app' artifact vs the 'platform' artifact), the country and the # expected file count. The count is what catches a partial copy, which the old # 'has any .app in it' test could not. $sourceTag = if ($sourceApps -eq $appApps) { 'app' } else { 'platform' } $mode = "$(if ($hasExtensions) { 'toolkit' } else { 'full' })-$sourceTag-$country-$($sourceAppFiles.Count)" $markerFile = Join-Path $staging '.albuild-symbols-mode' $isStaged = { if (-not (Test-Path -LiteralPath $staging)) { return $false } $marker = if (Test-Path -LiteralPath $markerFile) { "$(Get-Content -LiteralPath $markerFile -Raw -ErrorAction SilentlyContinue)".Trim() } else { '' } if ($marker -ne $mode) { return $false } # Wrapped in @(): an if that emits an empty array collapses to $null via the pipeline, # and .Count on $null then dies under Set-StrictMode. $have = @(Get-ChildItem -LiteralPath $staging -Filter '*.app' -File -ErrorAction SilentlyContinue) return ($have.Count -eq $sourceAppFiles.Count) } if ($sourceAppFiles.Count -eq 0) { # Nothing to stage - a sandbox whose platform carries no test toolkit. Not an error: # in that layout the business apps come from 'Extensions'. Write-ALbuildLog -Level Information "No first-party symbols to stage from '$sourceApps'." } else { if ($Force -or -not (& $isStaged)) { # Stage under a lock, and publish ATOMICALLY. Copying straight into $staging and # writing the marker afterwards leaves a window in which the folder exists and is # non-empty but incomplete. A concurrent build compiling against a half-copied # symbol folder fails with a misleading AL1022, or worse, compiles against a # first-party set that is silently missing apps. Same shape (and same lock naming) # as Get-BcArtifact's extraction. # # Two builds on one host hit this whenever they share an artifact; the runtime # factory makes it routine, because its workers are threads in a single process. $mutex = New-Object System.Threading.Mutex($false, (Get-ALbuildCacheLockName -Path $staging)) $held = $false try { try { $held = $mutex.WaitOne([TimeSpan]::FromMinutes(15)) } catch [System.Threading.AbandonedMutexException] { $held = $true } # prior holder crashed; we own it # Re-check under the lock: another build may have staged it while we waited. if ($Force -or -not (& $isStaged)) { # Short temp name, deliberately: BC artifacts already nest close to the # 260-char MAX_PATH limit, and a long ".staging-<guid>" sibling pushes the # deepest file over it. $temp = Join-Path (Split-Path -Parent $staging) ('.albtmp-' + [guid]::NewGuid().ToString('N').Substring(0, 8)) try { New-Item -ItemType Directory -Force -Path $temp | Out-Null $sourceAppFiles | Copy-Item -Destination $temp -Force # Marker goes in BEFORE the publish, so the folder is never visible # without it. Set-Content -LiteralPath (Join-Path $temp (Split-Path -Leaf $markerFile)) -Value $mode -Encoding UTF8 -NoNewline # Replacing an existing folder is where this used to go wrong. The # removal was allowed to fail silently and the rename then failed with # 'a file or directory with the same name already exists' - which the # caller logged as a warning and carried on, straight into a compile # against symbols that were not there. Two outcomes are legitimate: we # publish, or somebody else published the same set first. Anything # else has to be said out loud. if (Test-Path -LiteralPath $staging) { Remove-Item -LiteralPath $staging -Recurse -Force -ErrorAction SilentlyContinue } try { Move-ALbuildDirectory -Path $temp -Destination $staging $what = if ($hasExtensions) { 'test toolkit' } else { 'first-party' } Write-ALbuildLog -Level Information "Staged the $what symbols ($($sourceAppFiles.Count) app(s)) into '$staging'." } catch { if (& $isStaged) { Write-ALbuildLog -Level Information ("Another build published '$staging' first; using it " + "($($sourceAppFiles.Count) app(s), mode '$mode').") } else { throw ("Could not stage the first-party symbols for $($Artifact.Version) ($country) into " + "'$staging': $($_.Exception.Message) The folder is in the way and does not hold the " + "expected $($sourceAppFiles.Count) app(s); a build compiling against it would fail with " + 'AL1022 naming a Microsoft package. Remove it and run again.') } } } finally { if (Test-Path -LiteralPath $temp) { Remove-Item -LiteralPath $temp -Recurse -Force -ErrorAction SilentlyContinue } } } } finally { if ($held) { $mutex.ReleaseMutex() } $mutex.Dispose() } } # Only hand the compiler a folder that holds the whole set. 'Has some .app in it' was # the old test, and it is exactly what let a half-staged folder through: alc then # reports AL1022 for a Microsoft package, which reads like a broken artifact rather # than a race on the cache. if (& $isStaged) { $paths.Add($staging) } else { throw ("The first-party symbols for $($Artifact.Version) ($country) in '$staging' are not " + "the expected $($sourceAppFiles.Count) app(s) for mode '$mode'. Compiling against them " + 'would fail with AL1022 naming a Microsoft package. Re-run with -Force to re-stage.') } } } } } return $paths.ToArray() } |