html_element_toggle: toggle visibility of any HTML fragment ¶

Purpose: toggle visibility of one or more HTML fragments in a page

First, in the HTML page we need to sorround the fragment with a few tags; we do this with a Python helper:

from django.utils.translation import gettext_lazy as _
from django.utils.safestring import mark_safe

@register.filter
def html_element_toggle(html_fragment, start_hidden=True):
    if start_hidden:
        html = """
<div class="html_element_toggler">
    <a class="show" href="#"><i class="fa fa-eye"></i>&nbsp;%s</a>
    <a class="hide" href="#" style="display: none;"><i class="fa fa-eye-slash"></i>&nbsp;%s</a>
    <div class="data" style="display: none;">
        %s
    </div>
</div>
""" % (_('Show'), _('Hide'), html_fragment)
    else:
        html = """
<div class="html_element_toggler">
    <a class="show" href="#" style="display: none;"><i class="fa fa-eye"></i>&nbsp;%s</a>
    <a class="hide" href="#"><i class="fa fa-eye-slash"></i>&nbsp;%s</a>
    <div class="data">
        %s
    </div>
</div>
""" % (_('Show'), _('Hide'), html_fragment)

return mark_safe(html)

Then, after page loading is completed, we schedule toggling;

with either jQuery:

$(document).ready(function() {

    $('.html_element_toggler .show,.html_element_toggler .hide').on('click', function(event) {
        event.preventDefault();
        var target = $(event.target);
        target.parent().find('.show,.hide,.data').toggle();
    });

});

or vanilla Javascript:

function ready(fn) {
    if (document.readyState !== 'loading') {
        fn();
    } else {
        document.addEventListener('DOMContentLoaded', fn);
    }
}

function toggle(el) {
    if (el.style.display == 'none') {
        el.style.display = '';
    } else {
        el.style.display = 'none';
    }
}

function register_html_element_togglers() {
    let links = document.querySelectorAll('.html_element_toggler .show,.html_element_toggler .hide');
    for (let i = 0; i < links.length; ++i) {
        links[i].addEventListener('click', function(event) {
            event.preventDefault();
            let parent = event.target.parentNode;
            var children = parent.querySelectorAll('.show,.hide,.data');
            for (let j = 0; j < children.length; ++j) {
                toggle(children[j]);
            }
        });
    }
}

ready(register_html_element_togglers);

  How to serialize Django queryset.values() into json? ¶

Django's serializers can't handle a ValuesQuerySet.

However, you can serialize by using a standard json.dumps() and transforming your ValuesQuerySet to a list by using list().

If your set includes Django fields such as Decimals or UUIDs, you will need to pass in DjangoJSONEncoder.

Thus:

import json
from django.core.serializers.json import DjangoJSONEncoder
from django.http import HttpResponse

queryset = myModel.objects.filter(foo_icontains=bar).values('f1', 'f2', 'f3')
serialized_q = json.dumps(list(queryset), cls=DjangoJSONEncoder)

return HttpResponse(serialized_q, content_type='application/json')

Or just convert the ValuesQuerySet into a Python list, so you can further manipulate it:

import json
from django.core.serializers.json import DjangoJSONEncoder

queryset = myModel.objects.filter(foo_icontains=bar).values('f1', 'f2', 'f3')
data = list(queryset)

...

return JsonResponse(data, safe=False)

References:

• https://stackoverflow.com/questions/7650448/how-to-serialize-django-queryset-values-into-json#31994176

  NPM: Do I commit the package-lock.json file created by npm? ¶

post by "k0pernikus"

https://stackoverflow.com/questions/44206782/do-i-commit-the-package-lock-json-file-created-by-npm-5#56254478

Yes, you SHOULD:

• commit the package-lock.json.
• use npm ci instead of npm install when building your applications both on your CI and your local development machine

The npm ci workflow requires the existence of a package-lock.json.

A big downside of npm install command is its unexpected behavior that it may mutate the package-lock.json, whereas npm ci only uses the versions specified in the lockfile and produces an error

• if the package-lock.json and package.json are out of sync
• if a package-lock.json is missing.

Hence, running npm install locally, esp. in larger teams with multiple developers, may lead to lots of conflicts within the package-lock.json and developers to decide to completely delete the package-lock.json instead.

Yet there is a strong use-case for being able to trust that the project's dependencies resolve repeatably in a reliable way across different machines.

From a package-lock.json you get exactly that: a known-to-work state.

In the past, I had projects without package-lock.json / npm-shrinkwrap.json / yarn.lock files whose build would fail one day because a random dependency got a breaking update.

Those issue are hard to resolve as you sometimes have to guess what the last working version was.

• If you want to add a new dependency, you still run npm install {dependency}.
• If you want to upgrade, use either npm update {dependency} or npm install ${dependendency}@{version} and commit the changed package-lock.json.

If an upgrade fails, you can revert to the last known working package-lock.json.

  Schedule Linux server authomatic reboot once per week ¶

$ cat /etc/cron.d/reboot

# Once per week, on saturday morning
0 6 * * SAT     root    /sbin/shutdown --reboot +5

  Implicit UUID auto fields for primary keys in Django ¶

file app_name/fields.py

import uuid

from django.db.backends.base.operations import BaseDatabaseOperations
from django.db.models import AutoField, UUIDField

BaseDatabaseOperations.integer_field_ranges['UUIDField'] = (0, 0)


class UUIDAutoField(UUIDField, AutoField):
    def __init__(self, *args, **kwargs):
        kwargs.setdefault('default', uuid.uuid4)
        kwargs.setdefault('editable', False)
        super().__init__(*args, **kwargs)

file settings.py

DEFAULT_AUTO_FIELD = 'app_name.fields.UUIDAutoField'

References:

• https://stackoverflow.com/questions/77142041/implicit-uuid-auto-fields-for-primary-keys-in-django#77142298

  PostgreSQL: log slow queries ¶

Example:

# cat /etc/postgresql/14/main/conf.d/slow_query_logging.conf

log_min_duration_statement = 5000   # expressed in [ms]
logging_collector = on

# log_directory is an absolute path, or relative to the cluster data directory,
# which can be discovered as follows:
# sudo -u postgres psql -c "SHOW data_directory"
log_directory = '/var/log/postgresql/'

log_filename = 'postgresql-14-slow-queries.log'
log_file_mode = 0644

  Closing the DB connections after accessing Django ORM in secondary threads ¶

Since Django will create a new connection per thread when you access the ORM, and connections are left open even when the threads are terminated, we need to close the database connection explicitly when terminating a thread:

import threading
from django.db import connection


class MyThread(threading.Thread):

    def run(self):
        while keep_running:
            ...
            ...

        # When the device's communication loop exis, we still need to close the
        # database connection, since Django will create a new connection per thread
        # when you access the ORM, and connections are left open even when the threads
        # are terminated; see:
        # "Make sure you are closing the DB connections after accessing Django ORM in your threads":
        # https://james.lin.net.nz/2016/04/22/make-sure-you-are-closing-the-db-connections-after-accessing-django-orm-in-your-threads/
        connection.close()

If not, possible errors include:

• Too many connections
• "Database" is being accessed by other users

References:

• Make sure you are closing the DB connections after accessing Django ORM in your threads
• Database is being accessed by other users error when using ThreadPoolExecutor with Django

  Cutting the videos based on start and end time using ffmpeg ¶

ffmpeg -ss 00:01:00 -to 00:02:00 -i input.mp4 -c copy output.mp4

References:

https://stackoverflow.com/questions/18444194/cutting-the-videos-based-on-start-and-end-time-using-ffmpeg/42827058#42827058

  Python timezone test (with pytz and/or zoneinfo) ¶

Note that: Support for pytz is deprecated in Django 4.0 will be removed in Django 5.0

#!/usr/bin/env python3
import datetime
try:
    import pytz
except:
    pytz = None
try:
    import zoneinfo
except:
    zoneinfo = None


def get_system_tz():
    """
    "Get time zone information of the system in Python?"
    https://stackoverflow.com/questions/1111056/get-time-zone-information-of-the-system-in-python#51296984

    Example:

        print(get_system_tz())
        'CEST'

    """
    return datetime.datetime.now(datetime.timezone.utc).astimezone().tzname()


def is_tz_aware(dt):
    """
    "How to check if a datetime object is localized with pytz?"
    https://stackoverflow.com/questions/5802108/how-to-check-if-a-datetime-object-is-localized-with-pytz#27596917
    """
    return dt.tzinfo is not None and dt.tzinfo.utcoffset(dt) is not None


def print_dt(dt):
    tz_aware = is_tz_aware(dt)
    print("%s ... %s tzaware" % (dt.isoformat(), "is" if tz_aware else "is not"))


print("\nNaive:")
now_naive = datetime.datetime.now()
print_dt(now_naive)

print("\nApply system tz:")
now_tzaware = now_naive.astimezone()
print_dt(now_tzaware)

if pytz:
    print("\nApply specific tz using pytz:")
    now_pacific = now_naive.astimezone(pytz.timezone('US/Pacific'))
    print_dt(now_pacific)

if zoneinfo:
    print("\nApply specific tz using zoneinfo:")
    now_santo_domingo = now_naive.astimezone(zoneinfo.ZoneInfo(key='America/Santo_Domingo'))
    print_dt(now_santo_domingo)

Results:

Naive:
2023-07-25T16:00:26.781075 ... is not tzaware

Apply system tz:
2023-07-25T16:00:26.781075+02:00 ... is tzaware

Apply specific tz using pytz:
2023-07-25T07:00:26.781075-07:00 ... is tzaware

Apply specific tz using zoneinfo:
2023-07-25T10:00:26.781075-04:00 ... is tzaware