Fix settings

[poikilotherm]

What this PR does / why we need it:

• Fix errors in data migration SQL script (DDL migration script is much harder to test due to Flyway: disable DDL generation from EclipseLink #5871 / SQL Warnings on Deployment of WAR File to Test Instance #11074)
• Necessary fix for Pre/Post Publication workflows broken as of #11654 #11996

Which issue(s) this PR closes:

• Closes Pre/Post Publication workflows broken as of #11654 #11996

Special notes for your reviewer:
To be done.

Suggestions on how to test this:
Execute Migration DB Tests with mvn verify -PskipUnitTests -Dit.groups=migration
Edit: This PR absolutely needs to be tested with a real deployment on a real prod. database. Among other things, please test and confirm that any configured controlled vocabularies are still working, on the off chance that the :CVocConf setting is going to be treated differently. (L.A.)

Does this PR introduce a user interface change? If mockups are available, please link/include them here:
Nope

Is there a release notes update needed for this change?:
Nope

Additional documentation:
Introducing proper migration test here for the first time. Will need discussion and documentation. Docs may be beyond scope of this PR.
Happy to talk about the idea during tech hours.

[coveralls]

coverage: 24.174% (+0.004%) from 24.17%
when pulling 8ddd5b8 on 11996-fix-settings
into f2a250f on develop.

[qqmyers]

Update to allow numeric values in JSON looks fine, unit tests pass.

[landreev]

Just to have it on record here, both forms of the new json setting are now parsed properly, {"Rdata": "1048576", "default": "157286400"} and {"Rdata": 1048576, "default": 157286400}.

[landreev]

But please note that these ingest-related RestAssured tests are still failing:

	Test Result (8 failures / ±0)

    edu.harvard.iq.dataverse.api.DatasetsIT.testSemanticMetadataAPIs
    edu.harvard.iq.dataverse.api.FilesIT.testIngestSizeLimits
    edu.harvard.iq.dataverse.api.FilesIT.testDataSizeInDataverse
    edu.harvard.iq.dataverse.api.FilesIT.testAddFileToDatasetSkipTabIngest
    edu.harvard.iq.dataverse.api.FilesIT.testIngestWithAndWithoutVariableHeader
    edu.harvard.iq.dataverse.api.FilesIT.testUningestFileViaApi
    edu.harvard.iq.dataverse.api.FilesIT.test_006_ReplaceFileGoodTabular
    edu.harvard.iq.dataverse.api.FilesIT.testValidateDDI_issue6027

https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/view/change-requests/job/PR-12002/14/#showFailuresLink

(although I'm not 100% sure if the 1st one on the list is "ingest-related")

[landreev]

I'll take a closer look at the logs from the last Jenkins runs where all these failures happened.
As a very random thought - SettingsServiceBean is not a singleton; is there a scenario where we can have an instance of it that's stuck with a stale cached copy of a setting, when we keep changing it in a busy Dataverse instance?

[qqmyers]

FWIW: I created #12026 since I think we've seen the semantic web fail before.

[landreev]

You may want to completely disregard my guess about "stale settings".
But, could you tell me what's going on in this test: (starting line 1335 in FilesIT.java)

I mean, 987654321 is no longer an invalid value (?).
Is this passing simply because you are not giving ingest enough time to finish before asking for the datatable?
In Jenkins run number 14 this test failed further down, in line 1376. And that would in turn be consistent with the upload not succeeding on account of the ingest lock still being present. Locally, I was able to reproduce the failure in that spot only 2 out of 20 or so times. And that would be consistent with the ingest of this tiny file happening fast enough on most runs, such that the lock is gone by that point; and only failing intermittently when the ingest decides to take longer.
Again, just a guess/may be missing something... etc.

        // As the guides say, you MUST provide a string, not a JSON number.
        // That is, `"123"` in quotes rather than `123` with no quotes.
        // If you provide a number (no quotes) rather than a string,
        // all ingest will be disabled and you'll see an error in server.log
        // about how the system is misconfigured.
        String invalidNonString = """
{
  "default": 987654321
}
""";

        setLimit = UtilIT.setSetting(SettingsServiceBean.Key.TabularIngestSizeLimit, invalidNonString);
        setLimit.then().assertThat().statusCode(OK.getStatusCode());

        uploadFile = UtilIT.uploadFileViaNative(datasetId.toString(), pathToDataFile.toString(), apiToken);
        uploadFile.prettyPrint();
        uploadFile.then().assertThat()
                .statusCode(OK.getStatusCode())
                .body("data.files[0].label", equalTo("data-4.csv"));

        String fileId5 = JsonPath.from(uploadFile.body().asString()).getString("data.files[0].dataFile.id");

        getTabularFails = UtilIT.getFileDataTables(fileId5, apiToken);
        getTabularFails.prettyPrint();
        getTabularFails.then().assertThat()
                .statusCode(BAD_REQUEST.getStatusCode())
                .body("message", equalTo(BundleUtil.getStringFromBundle("files.api.only.tabular.supported")));

I did not investigate the other failures in similar depth.

[landreev]

(that makes me cautiously optimistic re: possibility that these are something finicky in the tests themselves, vs. an indication of any issues in the functionality)

[github-actions]

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:11996-fix-settings

ghcr.io/gdcc/configbaker:11996-fix-settings

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

[landreev]

I ran yet another Jenkins job by hand, and everything passed this time around. Merging.