build: make .tar.xz creation opt-out, fail if no xz

[rvagg]

During a recent release, the 8.x before last I think, we discovered belatedly that the macos release machine didn't have xz installed on it. This change was discussed (somewhere, but I know @MylesBorins was involved) to help resolve such oversights in future.

• Change XZ to HAS_XZ and make it 0=no and 1=yes
• Introduce SKIP_XZ as a possible user-supplied value, defaulting to 0=no
• Check for either HAS_XZ=1 or SKIP_XZ=1 on any release build that produces a tarball of some kind and fail if the system doesn't have xz and SKIP_XZ=1 hasn't been supplied.
• Only produce a .tar.xz if both HAS_XZ=1 and SKIP_XZ=0 (i.e. we have xz and haven't explicitly opted to skip)

This would get backported to active lines and we'd have to put a SKIP_XZ=1 for AIX builds (only).

I've tested the main permutations of this and it seems to work fine. I'd appreciate some experienced Makefile eyes though, @nodejs/build @bnoordhuis @joyeecheung?

[nodejs-github-bot]

@rvagg build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1664/pipeline

[refack]

LGTM % quastions

[refack]

/CC @nodejs/build-files

[rvagg]

OK, second commit on this PR, 94a2c9c8e5, implements the above behaviour suggested by @refack and the checks around the execution of xz is now simple again, although inverted from previous: ifeq ($(XZ), 1). That should make it easier to understand than the original logical-and within a single ifeq which is pretty opaque.

[refack]

Lite-CI (since it's not covered in CI anyway): https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/1678/

Reviewers, please 👍 if you are in favor of fast-tracking this PR.

[richardlau]

Lite-CI (since it's not covered in CI anyway): https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/1678/

Reviewers, please 👍 if you are in favor of fast-tracking this PR.

Is it worth asking @nodejs/releasers for a test release build?

[refack]

Is it worth asking @nodejs/releasers for a test release build?

https://ci-release.nodejs.org/job/iojs+release/3954/
https://ci-release.nodejs.org/job/iojs+release/3956 ✔️

[refack]

The ci-release job finished successfully, the output is at https://nodejs.org/download/nightly/v12.0.0-nightly20181122.2.94a2c9c8e5/ (no .xz for AIX)

[rvagg]

@refack you made a nightly with this with a weird date string: https://ci-release.nodejs.org/job/iojs+release/3956/ and now we have screwed up nightly version strings which won't parse with tools that expect a predictable format on this directory:

Please don't do that. Nightlies are strictly not for testing. I'm going to have to figure out if I can clean this one out now.

There's a bunch of good reasons we don't hand out ci-release access liberally, and the ability to make this kind of mess is one of them.

[refack]

Please don't do that. Nightlies are strictly not for testing. I'm going to have to figure out if I can clean this one out now.

My bad. What I meant to do was run a test build.

P.S. As a postmortem, we should add some validation code to assert our and our tools format assumptions (nodejs/build#1592)

[rvagg]

OK, so this should have failed on AIX but it didn't.

The config doesn't have SKIP_XZ=1 for AIX, and the output https://ci-release.nodejs.org/job/iojs+release/nodes=aix61-ppc64/3956/consoleFull doesn't mention anything about SKIP_XZ. It either hasn't invoked check-xz on there (not sure why it wouldn't) or HAS_XZ=1, which doesn't make sense because it also doesn't invoke xz..

[refack]

P.P.S. we also have this one - https://nodejs.org/download/nightly/v11.0.0-nightly201806050300f7c68cffa5de7f0099e3afa95093b0bbbe3c/

[rvagg]

Ah, so we have this in iojs+release:

if [[ "X${disttype}" != "Xrelease" ]]; then
  perl -pi -e "s/: release-only/:/g" Makefile
fi

That traces back to the addition of the REPLACEME stuff for API docs which is too strict. I tried to fix that back in #12958 but it got stalled with too much discussion, so the perl replacement has stuck and we're without the release-only checks entirely for release builds.. yay.

[refack]

Maybe it's because check-xz should be .PHONY?

[rvagg]

Proposed fix in #24575 so we can remove the perl release-only stripping.

[rvagg]

Discovered some problems, not sure why these weren't picked up with test builds in ci-release for the PR. #24841

[rvagg]

@nodejs/build FYI this is now in iojs+release for the ${ENV, var="OSTYPE"} = AIX61 build script:

It's building successfully now:

Results for this test going to https://nodejs.org/download/test/v12.0.0-test201812040ba0e0fd89/