Skip to Content
  2. Mailing Lists
  3. Contributors
  4. Re: 14.0 branches

Archives

Contributors

Re: 14.0 branches

Re: 14.0 branches

Re: 14.0 branches

As said, my only goal is to improve the OCA contribution workflow.
Of course, I don't want to force any specific installation method onto integrators workflows.

So let's try to keep the discussion focused.

To be a little bit more specific, here are some aspects that the new method improves:

Less Redundancy

What is in requirements.txt must be added manually and is redundant to addons manifests. oca_dependencies.txt is also redundant and the information can be found elsewhere.

Better testing of dependency declarations

We don't test external python dependencies declarations in manifests because we rely on a redundant, manually maintained requirements.txt
pylint-odoo helps but there are limits to what it can detect.

Also, we publish wheels for OCA addons because it is by far the easiest for newcomers (and veterans alike I should say) to discover dependencies (addons and external python dependencies). Bad dependencies makes for a suboptimal user experience for newcomers installing the wheels.

Reduced blast radius when addons are introduced with specific test requirements, or "exotic" test requirements break

When an addon has specific test requirements it has side effects in all repos that transitively depend on its repo, whether or not the modules are actually needed. When installing dependencies at the addon level instead of the repo level, we drastically reduce the blast radius of such things. Examples: server_environment that required a mandatory option in odoo.cfg. Or an obscure dependency of some rarely used addon in server-tools that suddenly breaks on PyPI and impacts all repos that transitively depend on server-tools because they all end up installing all requirements.txt of server-tools. When that happens, the impact is usually so wide that everyone has to rush to find solutions to unlock the situation. If the impact is contained to the addon and those that depend on it, that buys us more time to find good solutions.

Finer grained approach to declaring unmerged dependencies

With the optional test-requirements.txt, you can declare override dependencies to target unmerged individual addons like this:
This means you can reference several PRs of the same repos, or several addons of different PRs in another repo (I don't think one can do that with oca_dependencies.txt, where you reference whole repos).

[BTW, @Moises, to answer your question above you could use such references to your own repos. And it reduces the need to use git-aggregator too. - but I digress]

To close the list, I'd say dependency management is hard and relying on the broader python ecosystem to solve the problem can only help us in the long run (pip in particular is going to release a new sophisticated dependency resolver later this year). And Odoo is not particularly messy nor does it have anything very special in that matter.

So how can we progress?

For the reasons above, I'd like to move out of the status quo.
In the past the only arguments I had received were of the general class "I don't like it", but nothing really concrete.
So yeah, I guess I'm pushing a little bit in the hope of getting solid feedback.

No harm is done (there are no cross-repo dependencies in v14 branches yet :) and as said, we can easily push an update to .travis.yml.

And the good thing is that the feedback is coming:
- At this point I see Moises who says it relies on oca_dependencies.txt in its workflow.
- Daniel mentioned a documentation value of oca_dependencies.txt.

A few questions:
- @Moises, do you rely on requirement.txt too in your own workflow ?
- Do others rely on it ? (for instance, does Dooba need oca_dependencies.txt, requirements.txt in OCA repos ?)

If we conclude that oca_dependencies.txt and/or requirements.txt must be preserved in all repos in v14 (and I'm totally fine with that), we can then discuss how best to do it. Either manual with e.g. PIP mode on Odoo and OCA mode on OCB as Moises suggested - but that would be a burden for contributors to understand and maintain both approaches. Or try to generate oca_dependencies.txt and requirements.txt, which should be feasible at first sight, but that's more work. Or keep the status quo :/

Cheers,

-sbi


On Sun, Oct 11, 2020 at 7:06 PM Stéphane Bidoul <stephane.bidoul@acsone.eu> wrote:
On Sun, Oct 11, 2020 at 5:41 PM Ivan Todorovich <ivan.todorovich@druidoo.io> wrote:
Would testing with PIP also catch compatibility errors between modules that don’t depend on each other? Or errors with modules that are in the upstream chain of dependencies?

@Ivan the new method does not change what is installed on the test database. What changes is only the list of addons present in the addons path. So the scenarios you mention should work just like before.

-sbi
by Stéphane Bidoul - 07:15 - 11 Oct 2020

Reference

  • 14.0 branches
    Dear fellow contributors,

    The 14.0 branches are being created as I post this message.

    They are initialized from our new template repository that was created during the OCA Days sprint back in May [1]. This template is essentially a refreshed version of the linter configurations we have in 13.0. This new mechanism should make it easier to apply improvements across all repos in the future.

    Special thanks to Jairo Llopis for his work on this topic.

    I plan to provide a detailed walkthrough of all this during my OCA Days talk next week [6]. In the meantime, here are a few important things to note.

    1. The project description in README.md must be updated manually by PSCs.

    Since our project-level README were manually maintained and updated over a long period, it is difficult to reliably extract the variable content from them. So they are created afresh, and PSC are invited to update the repo description within the dedicated section of README.md. Please do not change the header and footer manually.

    2. Travis installs dependencies with pip, including addons of other repos

    This mechanism (activated by MQT_DEP=PIP in .travis.yml) does not use oca_dependencies.txt nor requirement.txt. It relies on __manifest__.py to discover dependencies from the 'depends' and 'external_dependencies' keys. Dependent addons are installed from the OCA wheelhouse [3], and python libraries are installed from PyPI.

    The main expected benefits are:
    - less redundancy (the manifests are enough to discover dependencies)
    - reduce rippling effects to unrelated repos when an addon or python library does not install or misbehaves, since only the dependencies really needed by a repo are installed

    If a PR depends on an unmerged addon or PR, create a file named test-requirements.txt at the repo root containing a line like this:


    This mechanism has been tested on several repos in 13.0 and should be reliable. In case of problem, mention me in the PR and/or create an issue in OCA/maintainer-quality-tools repo. Alternatively, you can restore the old behaviour by removing the MQT_DEP=PIP line from .travis.yml. For the curious, the code of the new mechanism is in the OCA/m-q-t repo [4]

    3. If you need local changes to the dotfiles let's discuss them

    There are variables in the dot files, including .travis.yml [2]. To update them, the best way is to install copier [5], run "copier update" from the repo root, and answer the questions.

    If you need other changes, you can apply them locally to resolve urgent situations, but that may make updates harder. So please open an issue in [1] to discuss if changes need to be made to the template.

    As usual, don't hesitate to let me know of any issue.

    That's all for now, folks. Happy migration!

    -sbi


    --
    Stéphane Bidoul | @SBidoul
    Acsone sa/nv | http://acsone.eu/ | +32 2 888 3120
    by Stéphane Bidoul - 09:21 - 8 Oct 2020