• 3 Posts
  • 25 Comments
Joined 2 years ago
cake
Cake day: June 17th, 2023

help-circle
  • If you needs are simple, write a simple playbook using the proxmox ansible module https://docs.ansible.com/ansible/latest/collections/community/general/proxmox_kvm_module.html

    Terraform/Opentofu provides more advanced stuff but then you have to worry about persistent state storage, the clunky DSL… used it when acsolutely needed, you can do 90% of this stuff with the proxmox ansible module.

    If you need to make your playbook less verbose, move the logic to a role so that you can configure your VMs from a few lines in the playbook/host_vars. Mine looks like this (it’s for libvirt and not proxmox, but the logic is the same)

    # playbook.yml
    - hosts: hypervisor.example.org
      roles:
        - libvirt
    
    # host_vars/hypervisor.example.org.yml
    libvirt_vms:
      - name: vm1.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm1.example.org.xml"
        state: running
        autostart: yes
      - name: vm2.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm2.example.org.xml"
        autostart: no
      - name: vm3.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm3.example.org.xml"
        autostart: no
      - name: vm4.example.org
        xml_file: "{{ playbook_dir }}/data/libvirt/vm4.example.org.xml"
        autostart: no
        disk_size: 100G
    


    • Ever tested restoring those backups? Do you have the exact procedure written down? Does it still work? If the service gets compromised/data corrupted on sunday, and your backup runs, do you still have a non-compromised backup and how old is it?
    • How timely can you deal with security fixes, and how will you be alerted that a security fix is available?
    • How do you monitor your services for resource availability, errors in logs, security events?
    • How much downtime is acceptable for routine maintenance, and for incidents?
    • Do you have tooling to ensure you can redeploy the exact same configuration to another host?
    • How do you test upgrades before pushing them to production?

    Not saying this is impossible, you just need to have these questions in mind, and the answers written down before you start charging people for the service, and have the support infrastructure ready.

    Or you can just provide the service for free, best-effort without guarantees.

    I do both (free services for a few friends, paid by customers at $work, small team). Most of the time it’s smooth riding but it needs preparation (and more than 1 guy to handle emergencies - vacations, bus factor and all that).

    For the git service I can recommend gitea + gitea-actions (I run the runners in podman). Gitlab has more features but it can be overwhelming if you don’t need them, and it requires more resources.


  • vegetaaaaaaa@lemmy.worldtoSelfhosted@lemmy.worldVersion Dashboard
    link
    fedilink
    English
    arrow-up
    1
    ·
    edit-2
    2 months ago

    I use RSS feeds, bump version numbers when a new release is out, git commit/push and the CI does the rest (or I’ll run the ansible playbook manually).

    I do check the release notes for breaking changes, and sometimes hold back updates for some time (days/weeks) when the release affects a “critical” feature, or when config tweaks are needed, and/or run these against a testing/staging environment first.




  • Right, I just spent 10 minutes looking for documentation that doesn’t involve shitty expensive SaaS/PaaS, couldn’t find anything. That disqualifies it for me as well, sorry for wasting your time.

    I’ll keep watching this thread, relevant to my interests as well. At work we let ansible (in pull mode) handle the Linux fleet, Android we don’t have enough devices to bother, and are looking towards jamf for macs. But I’d love to find a FOSS solution too, our requirements are simple enough (as you said install/remove stuff, change basic settings)


  • My prod and testing environments are 2 libvirt VMs on the same hypervisor. They run the same services, deployed and managed by ansible. The testing VM just gets less disk/CPU/RAM resources, and is powered off most of the time. Simple config changes? Straight to prod. New feature, risky change? Testing first.







    • simple: rsyslog: all local logs to a central syslog file (using the imfile module), all syslogsfrom all server to a central rsyslog server (over TCP/SSL, example here). Use lnav or something similar to consume the logs
    • more complex, resource-heavy: Graylog Open as a replacement for the central rsyslog server, setup pipelines/alerts/whatever… Currently considering replacing my Graylog instance with Wazuh but I don’t know yet if it will be able to replace it completely for me

  • security

    with containers, software maintainers also need to keep their image up-to-date with latest security fixes (most of them don’t) - whereas these are usually handled by unattended-upgrades or similar in a VM. Then put out a new release and expect users to upgrade ASAP. Or rebuild and encourage redeploying the latest image every day or so, which is bad for other reasons (no warning for breaking changes, the software must be tested thoroughly after every commit to master).

    In short this adds the burden of proper OS/image maintenance for developers, something usually handled by distro maintainers.

    trivy is helpful in assessing the maintenance/vulnerability level of OCI images.