Brainstorm's snippets (20/258)


  Django & HTTPS tricks
https
  PostgreSQL ODBC connection from Ubuntu
odbc
  CookiesDirective for Privacy Policy
javascript
  Docker notes
docker
  django-imagekit: Generate predictable filenames in cache
imagekit
  Send an HTTP request using Telnet
telnet, http
  Ngrok references
ngrok
  Building Python 3 on embedded Linux
linux, python3
  django-parler: Make existing fields translatable
parler
  django-imagekit: How to cleanup cache backend and remove generated thumbnails from storage
thumbnail, django-imagekit
  QuickTime tips
quicktime
  Homebrew Cheatsheet
homebrew
  Howto annotate a QuerySet with calculated properties and use them in ModelAdmin for filtering and sorting
admin, query
  Enable/Disable buttons based on HTML page status
html
  Understand Group by in Django with SQL
django, orm
  Make sure request.is_ajax() return True when sending a POST request from javascript
post
  Google maps alternatives
map
  How to enable Screen Sharing on Macs via Terminal
mac
  Easy Printing
python, print
  Redirect to an arbitrary URL After Saving in Django Admin
django, admin, redirect

  Django & HTTPS tricks

Testing HTTPS in development

sudo pip install django-sslserver

INSTALLED_APPS = [
    ...
    "sslserver",
    ...
]

then:

python manage.py runsslserver 0.0.0.0:8888

Note that for some reason port 8000 hangs; use 8888 instead.

Credits: https://stackoverflow.com/questions/8023126/how-can-i-test-https-connections-with-django-as-easily-as-i-can-non-https-connec#51384868

With NGINX

In your Django settings, set the SECURE_PROXY_SSL_HEADER setting:

SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

Then, you need Nginx to set the custom header in the reverse proxy. In the Nginx site settings:

location / {
    # ...
    proxy_set_header X-Forwarded-Proto $scheme;
}

This way request.scheme == 'https' and request.is_secure() returns True. request.build_absolute_uri() returns https://... and so on...

Credits: https://stackoverflow.com/questions/8153875/how-to-deploy-an-https-only-site-with-django-nginx#19637196

  PostgreSQL ODBC connection from Ubuntu

Install the basic config tools for the UNIX ODBC:

sudo apt-get install unixodbc unixodbc-dev

Install ODBC drivers for PostgreSQL:

sudo apt-get install odbc-postgresql

Configure the PostgreSQL and ODBC driver.

File /etc/odbcinst.ini:

[PostgreSQL ANSI]
Description=PostgreSQL ODBC driver (ANSI version)
Driver=psqlodbca.so
Setup=libodbcpsqlS.so
Debug=0
CommLog=1
UsageCount=1

[PostgreSQL Unicode]
Description=PostgreSQL ODBC driver (Unicode version)
Driver=psqlodbcw.so
Setup=libodbcpsqlS.so
Debug=0
CommLog=1
UsageCount=1

File /etc/odbc.ini:

[Gallery]
Driver = PostgreSQL Unicode
Description = ODBC connection to Gallery via PostgreSQL Unicode
Trace = No
#Trace = Yes
#TraceFile = /tmp/sql.log
Servername = localhost
Port = 5432
#Protocol = 8.4
Database = gallery
UserName = ****
Password = ****
#ReadOnly = No

Test the ODBC to PostgreSQL connection by running the isql command, which reads the /etc/odbc.ini file:

$ isql -v gallery <username> <password>
+---------------------------------------+
| Connected!                            |
|                                       |
| sql-statement                         |
| help [tablename]                      |
| quit                                  |
|                                       |
+---------------------------------------+

SQL> select count(*) from users_user;
+---------------------+
| count               |
+---------------------+
| 3                   |
+---------------------+
SQLRowCount returns 1
1 rows fetched

  CookiesDirective for Privacy Policy

In your base.html add:

<script src="{% static 'js/jquery.cookiesdirective.js' %}"></script>

...

<script>

    $(document).ready(function() {

        /*
        $.cookiesDirective({
            privacyPolicyUri: 'my-privacy-policy.html',
            explicitConsent: false
        });

        $(".privacyPolicyButton").click(function (e) {
            e.preventDefault();
            $("#privacyPolicy").modal('show');
        });
        */


        // Cookie setting script wrapper
        var cookieScripts = function () {
            /*
            // Internal javascript called
            console.log("Running");

            // Loading external javascript file
            $.cookiesDirective.loadScript({
                uri:'external.js',
                appendTo: 'eantics'
            });
            */

            $("#privacyPolicyTag").animate({
              opacity:'1'
            },2000);
        };

        $.cookiesDirective({
            privacyPolicyUri: '/pc-policy/privacy-policy/',
            explicitConsent: false,
            scriptWrapper: cookieScripts,
            position: 'bottom',
            message: '{% trans "This site uses cookies. More details can be found in our" %}',
            privacyPolicyTitle: '{% trans "Privacy Policy" %}',
            dismissButtonTitle: '{% trans "Do not show this message again" %}'
        });

        /*
        $(".privacyPolicyButton").click(function (e) {
            e.preventDefault();
            //$("#privacyPolicyDialog").modal('show');
            $("#privacyPolicyDialog").dialog({
                width: 500,
                height: 400
            });
        });
        */

    });

</script>

where jquery.cookiesdirective.js has been adapted from: https://github.com/krazyjakee/CookiesDirective

/* Cookies Directive - The rewrite. Now a jQuery plugin
 * Version: 2.0.1
 * Author: Ollie Phillips
 * 24 October 2013
 *
 * More information at http://cookiesdirective.com
 */

;(function($) {
  $.cookiesDirective = function(options) {

    // Default Cookies Directive Settings
    var settings = $.extend({
      //Options
      explicitConsent: true,
      position: 'top',
      duration: 10,
      limit: 0,
      message: null,
      dismissButtonTitle: 'Do not show this message again',
      privacyPolicyTitle: 'Privacy Policy',
      cookieScripts: null,
      privacyPolicyUri: 'privacy.html',
      scriptWrapper: function(){},
      // Styling
      fontFamily: 'helvetica',
      fontColor: '#FFFFFF',
      fontSize: '13px',
      backgroundColor: '#000000',
      backgroundOpacity: '80',
      //linkColor: '#CA0000',
      linkColor: '#ff0',
      impliedSubmitColor: '#000'
    }, options);

    // Perform consent checks
    if(!getCookie('cookiesDirective')) {
      if(settings.limit > 0) {
        // Display limit in force, record the view
        if(!getCookie('cookiesDisclosureCount')) {
          setCookie('cookiesDisclosureCount',1,1);
        } else {
          var disclosureCount = getCookie('cookiesDisclosureCount');
          disclosureCount ++;
          setCookie('cookiesDisclosureCount',disclosureCount,1);
        }

        // Have we reached the display limit, if not make disclosure
        if(settings.limit >= getCookie('cookiesDisclosureCount')) {
          disclosure(settings);
        }
      } else {
        // No display limit
        disclosure(settings);
      }

      // If we don't require explicit consent, load up our script wrapping function
      if(!settings.explicitConsent) {
        settings.scriptWrapper.call();
      }
    } else {
      // Cookies accepted, load script wrapping function
      settings.scriptWrapper.call();
    }
  };

  // Used to load external javascript files into the DOM
  $.cookiesDirective.loadScript = function(options) {
    var settings = $.extend({
      uri:    '',
      appendTo:   'body'
    }, options);

    var elementId = String(settings.appendTo);
    var sA = document.createElement("script");
    sA.src = settings.uri;
    sA.type = "text/javascript";
    sA.onload = sA.onreadystatechange = function() {
      if ((!sA.readyState || sA.readyState == "loaded" || sA.readyState == "complete")) {
        return;
      }
    };
    switch(settings.appendTo) {
      case 'head':
        $('head').append(sA);
          break;
      case 'body':
        $('body').append(sA);
          break;
      default:
        $('#' + elementId).append(sA);
    }
  };

  // Helper scripts
  // Get cookie
  var getCookie = function(name) {
    var nameEQ = name + "=";
    var ca = document.cookie.split(';');
    for(var i=0;i < ca.length;i++) {
      var c = ca[i];
      while (c.charAt(0)==' ') c = c.substring(1,c.length);
      if (c.indexOf(nameEQ) === 0) return c.substring(nameEQ.length,c.length);
    }
    return null;
  };

  // Set cookie
  var setCookie = function(name,value,days) {
    var expires = "";
    if (days) {
      var date = new Date();
      date.setTime(date.getTime()+(days*24*60*60*1000));
      expires = "; expires="+date.toGMTString();
    }
    document.cookie = name+"="+value+expires+"; path=/";
  };

  // Detect IE < 9
  var checkIE = function(){
    var version;
    if (navigator.appName == 'Microsoft Internet Explorer') {
          var ua = navigator.userAgent;
          var re = new RegExp("MSIE ([0-9]{1,}[\\.0-9]{0,})");
          if (re.exec(ua) !== null) {
              version = parseFloat(RegExp.$1);
      }
      if (version <= 8.0) {
        return true;
      } else {
        if(version == 9.0) {
          if(document.compatMode == "BackCompat") {
            // IE9 in quirks mode won't run the script properly, set to emulate IE8
            var mA = document.createElement("meta");
            mA.content = "IE=EmulateIE8";
            document.getElementsByTagName('head')[0].appendChild(mA);
            return true;
          } else {
            return false;
          }
        }
        return false;
      }
      } else {
      return false;
    }
  };

  // Disclosure routines
  var disclosure = function(options) {
    var settings = options;
    settings.css = 'fixed';

    // IE 9 and lower has issues with position:fixed, either out the box or in compatibility mode - fix that
    if(checkIE()) {
      settings.position = 'top';
      settings.css = 'absolute';
    }

    // Any cookie setting scripts to disclose
    var scriptsDisclosure = '';
    if (settings.cookieScripts) {
      var scripts = settings.cookieScripts.split(',');
      var scriptsCount = scripts.length;
      var scriptDisclosureTxt = '';
      if(scriptsCount>1) {
        for(var t=0; t < scriptsCount - 1; t++) {
           scriptDisclosureTxt += scripts[t] + ', ';
        }
        scriptsDisclosure = ' We use ' +  scriptDisclosureTxt.substring(0,  scriptDisclosureTxt.length - 2) + ' and ' + scripts[scriptsCount - 1] + ' scripts, which all set cookies. ';
      } else {
        scriptsDisclosure = ' We use a ' + scripts[0] + ' script which sets cookies.';
      }
    }

    // Create overlay, vary the disclosure based on explicit/implied consent
    // Set our disclosure/message if one not supplied
    var html = '';
    html += '<div id="epd">';
    html += '<div id="cookiesdirective" style="position:'+ settings.css +';'+ settings.position + ':-300px;left:0px;width:100%;';
    html += 'height:auto;background:' + settings.backgroundColor + ';opacity:.' + settings.backgroundOpacity + ';';
    html += '-ms-filter: “alpha(opacity=' + settings.backgroundOpacity + ')”; filter: alpha(opacity=' + settings.backgroundOpacity + ');';
    html += '-khtml-opacity: .' + settings.backgroundOpacity + '; -moz-opacity: .' + settings.backgroundOpacity + ';';
    html += 'color:' + settings.fontColor + ';font-family:' + settings.fontFamily + ';font-size:' + settings.fontSize + ';';
    html += 'text-align:center;z-index:1000;">';
    html += '<div style="position:relative;height:auto;width:90%;padding:10px;margin-left:auto;margin-right:auto;">';

    if(!settings.message) {
      if(settings.explicitConsent) {
        // Explicit consent message
        // settings.message = 'This site uses cookies. Some of the cookies we ';
        // settings.message += 'use are essential for parts of the site to operate and have already been set.';

        settings.message = "Questo sito o gli strumenti terzi da questo utilizzati si avvalgono di cookie necessari al funzionamento ed utili alle finalità illustrate nella cookie policy.<br />Chiudendo questo banner, scorrendo questa pagina, cliccando su un link o proseguendo la navigazione in altra maniera, acconsenti all’uso dei cookie.";

      } else {
        // Implied consent message
        //settings.message = 'We have placed cookies on your computer to help make this website better.';
        settings.message = "Questo sito o gli strumenti terzi da questo utilizzati si avvalgono di cookie necessari al funzionamento ed utili alle finalità illustrate nella cookie policy.<br />Chiudendo questo banner, scorrendo questa pagina, cliccando su un link o proseguendo la navigazione in altra maniera, acconsenti all’uso dei cookie.";
      }
    }
    html += settings.message;

    // Build the rest of the disclosure for implied and explicit consent
    /* !!!
    if(settings.explicitConsent) {
      // Explicit consent disclosure
      html += scriptsDisclosure + 'You may delete and block all cookies from this site, but parts of the site will not work.';
      html += 'To find out more about cookies on this website, see our <a style="color:'+ settings.linkColor + ';font-weight:bold;';
      html += 'font-family:' + settings.fontFamily + ';font-size:' + settings.fontSize + ';" href="'+ settings.privacyPolicyUri + '">privacy policy</a>.<br/>';
      html += '<div id="epdnotick" style="color:#ca0000;display:none;margin:2px;"><span style="background:#cecece;padding:2px;">You must tick the "I accept cookies from this site" box to accept</span></div>';
      html += '<div style="margin-top:5px;">I accept cookies from this site <input type="checkbox" name="epdagree" id="epdagree" />&nbsp;';
      html += '<input type="submit" name="explicitsubmit" id="explicitsubmit" value="Continue"/><br/></div></div>';



    } else {
      // Implied consent disclosure
      //html += scriptsDisclosure + ' More details can be found in our <a style="color:'+ settings.linkColor + ';';
      html += scriptsDisclosure + '<br />Ulteriori informazioni sono specificate nella nostra <a class="privacyPolicyButton" style="color:'+ settings.linkColor + ';';
      html += 'font-weight:bold;font-family:' + settings.fontFamily + ';font-size:' + settings.fontSize + ';" href="'+ settings.privacyPolicyUri + '">Privacy Policy</a>.';
      //html += '<div style="margin-top:5px;"><input type="submit" name="impliedsubmit" id="impliedsubmit" value="Do not show this message again"/></div></div>';
      html += '<div style="margin-top:5px;"><input type="submit" name="impliedsubmit" id="impliedsubmit" style="color:'+ settings.impliedSubmitColor +';" ' + 'value="Non mostrare più questo messaggio"/></div></div>';
    }
    */

    html += ' <a class="privacyPolicyButton" style="color:#ff0;font-weight:bold;font-family:helvetica;font-size:13px;" href="' + settings.privacyPolicyUri + '">' +
        settings.privacyPolicyTitle + '</a>';
    html += '<div style="margin-top:5px;"><input type="submit" name="impliedsubmit" id="impliedsubmit" value="' + settings.dismissButtonTitle + '"/></div></div>';

    html += '</div></div>';
    $('body').append(html);

    // Serve the disclosure, and be smarter about branching
    var dp = settings.position.toLowerCase();
    if(dp != 'top' && dp!= 'bottom') {
      dp = 'top';
    }
    var opts = { in: null, out: null};
    if(dp == 'top') {
      opts.in = {'top':'0'};
      opts.out = {'top':'-300'};
    } else {
      opts.in = {'bottom':'0'};
      opts.out = {'bottom':'-300'};
    }

    // Start animation
    $('#cookiesdirective').animate(opts.in, 1000, function() {
      // Set event handlers depending on type of disclosure
      if(settings.explicitConsent) {
        // Explicit, need to check a box and click a button
        $('#explicitsubmit').click(function() {
          if($('#epdagree').is(':checked')) {
            // Set a cookie to prevent this being displayed again
            setCookie('cookiesDirective',1,365);
            // Close the overlay
            $('#cookiesdirective').animate(opts.out,1000,function() {
              // Remove the elements from the DOM and reload page
              $('#cookiesdirective').remove();
              location.reload(true);
            });
          } else {
            // We need the box checked we want "explicit consent", display message
            $('#epdnotick').css('display', 'block');
          }
        });
      } else {
        // Implied consent, just a button to close it
        $('#impliedsubmit').click(function() {
          // Set a cookie to prevent this being displayed again
          setCookie('cookiesDirective',1,365);
          // Close the overlay
          $('#cookiesdirective').animate(opts.out,1000,function() {
            // Remove the elements from the DOM and reload page
            $('#cookiesdirective').remove();
          });
        });
      }

      // Set a timer to remove the warning after 'settings.duration' seconds
      setTimeout(function(){
        $('#cookiesdirective').animate({
          opacity:'0'
        },2000, function(){
          $('#cookiesdirective').css(dp,'-300px');
        });
        $("#privacyPolicyTag").animate({
          opacity:'1'
        },2000);
      }, settings.duration * 1000);
    });
  };
})(jQuery);

  Docker notes


OBSOLETE! see https://github.com/morlandi/DockerDev

Cheatsheet

docker container ls -a List all containers
docker container ls -a -q List all containers in "quiet" mode
docker rm $(docker container ls -a -q) Remove all containsers
docker image ls -a List images
docker rmi IMAGENAME Remove image

Running an instantaneous Ubuntu Machine

System Message: WARNING/2 (<string>, line 23)

Title underline too short.

Running an instantaneous Ubuntu Machine
--------------------------------------

We will run a Ubuntu machine working with an interactive docker container.

This is the very ground level knowledge on how using Docker as a user environment, rather than a standalone container with an app running in it.

docker run --interactive --tty ubuntu /bin/bash

What we did:

  • we created a ubuntu machine with a random name

or:

docker run --name ubuntu -v /Users/morlandi/tmp/sources:/src -t -i ubuntu /bin/bash

What we did:

  • we created a ubuntu machine named "ubuntu"
  • with a volumes attached to it, directly from our machine
  • we mapped the volume using the "-v" flag [local]:[destination],

The lifetime of a container is the life time of its single main process.

Time to check docker, see what happened with it:

docker ps -a

CONTAINER ID        IMAGE               COMMAND             NAMES
e41423f7856b        ubuntu              "/bin/bash"         ubuntu
ed5887209364        ubuntu              "/bin/bash"         cool_proskuriakova

Finally, we cleanup the images:

docker rm -f cool_proskuriakova
docker rm -f ubuntu

Mac docker volume mount using osxfs

Docker Desktop for Mac started using osxfs for supporting volume mounting on MacOS.

The following command mounts the ~/Desktop directory to the docker container:

docker run -it -v ~/Desktop:/Desktop ubuntu bash

Proof:

from the container:

root@71f6e65abc6e:/# ls /
bin  boot  Desktop  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

root@71f6e65abc6e:/# ls -l /Desktop/

    -rw-r--r-- 1 root root 127138 Jul 17 14:52 '/Desktop/Screenshot 2020-07-17 at 16.51.58.png'
    -rw-r--r-- 1 root root 648435 Jul 17 15:47 '/Desktop/Screenshot 2020-07-17 at 17.47.21.png'
    -rw-r--r-- 1 root root 249486 Jul 17 16:15 '/Desktop/Screenshot 2020-07-17 at 18.15.15.png'
    -rw-r--r-- 1 root root 821404 Jul 18 08:57 '/Desktop/Screenshot 2020-07-18 at 10.57.38.png'

from the host:

$ ls -l ~/Desktop

    -rw-r--r--@ 1 morlandi  staff  127138 Jul 17 16:52 Desktop/Screenshot 2020-07-17 at 16.51.58.png
    -rw-r--r--@ 1 morlandi  staff  648435 Jul 17 17:47 Desktop/Screenshot 2020-07-17 at 17.47.21.png
    -rw-r--r--@ 1 morlandi  staff  249486 Jul 17 18:15 Desktop/Screenshot 2020-07-17 at 18.15.15.png
    -rw-r--r--@ 1 morlandi  staff  821404 Jul 18 10:57 Desktop/Screenshot 2020-07-18 at 10.57.38.png

  django-imagekit: Generate predictable filenames in cache

file settings.py:

#
#  ImageKit
#

#IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY = 'imagekit.cachefiles.strategies.Optimistic'

# Generate predictable filenames in cache
#IMAGEKIT_SPEC_CACHEFILE_NAMER = 'imagekit.cachefiles.namers.source_name_as_path'
IMAGEKIT_SPEC_CACHEFILE_NAMER = 'main.imagekit_overrides.source_name_as_path'

where file main.imagekit_overrides.py is:

# adapted from "imagekit.cachefiles.namers.source_name_as_path.py"

from django.conf import settings
import os
from imagekit.utils import format_to_extension, suggest_extension


def my_get_hash(generator):
    name = ''
    try:
        processor = generator.processors[0]
        width = processor.width
        height = processor.height
        name = '%dx%d' % (width, height)
    except Exception as e:
        name = generator.get_hash()
    return name


def source_name_as_path(generator):
    """
    A namer that, given the following source file name::

        photos/thumbnails/bulldog.jpg

    will generate a name like this::

        /path/to/generated/images/photos/thumbnails/bulldog/5ff3233527c5ac3e4b596343b440ff67.jpg

    where "/path/to/generated/images/" is the value specified by the
    ``IMAGEKIT_CACHEFILE_DIR`` setting.

    """
    source_filename = getattr(generator.source, 'name', None)

    if source_filename is None or os.path.isabs(source_filename):
        # Generally, we put the file right in the cache file directory.
        dir = settings.IMAGEKIT_CACHEFILE_DIR
    else:
        # For source files with relative names (like Django media files),
        # use the source's name to create the new filename.
        dir = os.path.join(settings.IMAGEKIT_CACHEFILE_DIR,
                           os.path.splitext(source_filename)[0])

    ext = suggest_extension(source_filename or '', generator.format)
    #return os.path.normpath(os.path.join(dir, '%s%s' % (generator.get_hash(), ext)))
    return os.path.normpath(os.path.join(dir, '%s%s' % (my_get_hash(generator), ext)))

  Send an HTTP request using Telnet

$ telnet brainstorm.it 80
Trying 116.203.5.135...
Connected to brainstorm.it.
Escape character is '^]'.

GET / HTTP/1.0
Host: brainstorm.it
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Tue, 07 Jul 2020 06:48:29 GMT
Content-Type: text/html
Content-Length: 178
Connection: close
Location: https://brainstorm.it/

<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
Connection closed by foreign host.

For https-only reachable url:

$ ncat --ssl brainstorm.it 443
GET / HTTP/1.0
Host: brainstorm.it
...
...
                });
            });
        })(jQuery);
    </script>

  </body>
</html>

  Building Python 3 on embedded Linux

curl -O https://www.python.org/ftp/python/3.8.4/Python-3.8.4rc1.tgz
tar xzvf Python-3.8.4rc1.tgz
./configure
make
make install

References:

Install virtualenvwrapper (optional)

pip3 install virtualenvwrapper

Add this to ~/.profile:

alias ll='/bin/ls -lh'

export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3
export WORKON_HOME=~/.virtualenvs
#mkdir -p $WORKON_HOME
source /usr/local/bin/virtualenvwrapper.sh

To create a new virtualenv:

mkvirtualenv webapp --seeder=pip

  django-parler: Make existing fields translatable

Procedura per traduzione tabella Product

  • I campi name, slug e description (pre-esistenti) devono essere tradotti

  • Commentato ProductAdmin (sostituito con pass)

  • Rinominati i campi field --> old_field in Product

  • Creata migrazione "product_rename_old_field"

  • Product derivato da TranslatableBaseModel anziche' BaseModel

  • introdotti i campi "translations":

    translations = TranslatedFields(
        name = models.CharField(max_length=1024, verbose_name=_("Name")),
        slug = models.SlugField(_("Slug"), max_length=255, unique=True),
        description = models.TextField(blank=True, verbose_name=_("Description")),
    
        meta = {'unique_together': [('language_code', 'slug')]},
    )
    
  • Creata migrazione "product_rename_add_translation_fields"

  • Fix migrazione precedente:

    migrations.CreateModel(
        name='Product',
        ...
        bases=(parler.models.TranslatableModelMixin, models.Model),
    ),
    
  • Creata data migration "product_migrate_translatable_fields":

    from django.db import migrations
    from django.conf import settings
    
    
    def forwards_func(apps, schema_editor):
        MyModel = apps.get_model('backend', 'Product')
        MyModelTranslation = apps.get_model('backend', 'ProductTranslation')
    
        for object in MyModel.objects.all():
            MyModelTranslation.objects.create(
                master_id=object.pk,
                language_code=settings.LANGUAGE_CODE,
                name=object.old_name,
                slug=object.old_slug,
                description=object.old_description,
            )
    
    
    class Migration(migrations.Migration):
    
        dependencies = [
            ('backend', '0003_product_add_translation_fields'),
        ]
    
        operations = [
            migrations.RunPython(forwards_func),
        ]
    
  • Commentati i campi "old_field" e creata migrazione "product_remove_old_fields"

  • ProductAdmin derivata da TranslatableBaseModelAdmin anziche' BaseModelAdmin

  • In ProductAdmin, utilizzato metodo get_prepopulated_fields() anziche' attributo prepopulated_fields

  • Eseguite migrazioni

  django-imagekit: How to cleanup cache backend and remove generated thumbnails from storage

Best effort to regenerate the thumbnail when the associated image is updated

In a django Model, I keep an ImageField "image" and an associated "thumbnail", based on from imagekit.models.ImageSpecField.

I might need to rebuild the image, keeping the same filename for the bitmap.

Obviously, the thumbnail needs to be updated as well; my first attempt was:

self.thumbnail.generate(force=True)

but that's not enough; despite the force=True parameter, the thumbnail remains unchanged.

After some trials and errors, I ended up with the following workaround:

class MyModel(model.Model):

    ...

    image = models.ImageField(_('Image'), null=True, blank=True, upload_to=get_acquisition_media_file_path)
    thumbnail = ImageSpecField(source='image', processors=[ResizeToFill(200, 100)], format='PNG')

    ...

    def save_image_from_data(self):

        # Create a new random image
        from random import randint
        rgb = (randint(0, 255), randint(0, 255), randint(0, 255))
        image_size = (200, 200)
        image = PILImage.new('RGB', image_size, rgb)
        with io.BytesIO() as output:
            image.save(output, format="PNG")
            contents = output.getvalue()

        # Best effort to remove the thumbnail, if any
        try:
            file = self.thumbnail.file
        #except FileNotFoundError:
        except:
            pass
        try:
            cache_backend = self.thumbnail.cachefile_backend
            cache_backend.cache.delete(cache_backend.get_key(file))
            #self.thumbnail.storage.delete(file)
            self.thumbnail.storage.delete(file.name)
        except:
            pass

        # Save image
        self.image = ContentFile(contents, "acquisition_%d.png" % self.position)
        self.save()

        # Regenerate thumbnail
        self.thumbnail.generate(force=True)

        return True

Thumbnails cleanup

ImageKit has a bug where files are cached and not deleted right away:

https://github.com/matthewwithanm/django-imagekit/issues/229#issuecomment-315690575

An interesting solution to the is the following.

UNTESTED: I didn't try this yet in a real situation

from django.db.models.signals import post_delete

    @staticmethod
    def post_delete(sender, instance, **kwargs):
        for field in ['thumbnail', ]:
            field = getattr(instance, field)
            try:
                file = field.file
            except FileNotFoundError:
                pass
            else:
                cache_backend = field.cachefile_backend
                cache_backend.cache.delete(cache_backend.get_key(file))
                field.storage.delete(file)
        instance.file.delete(save=False)

post_delete.connect(MyModel.post_delete, sender=MyModel)

References

Manually clear the cache

because in ImageKit 4.0 django cache backend is configured to preserve the state of images forever then you need to clear it also.

from imagekit.utils import get_cache
get_cache().clear()

  QuickTime tips

How to stop a screen recording:

Control + Command + Esc

How to Edit Video Using QuickTime Player for Mac

Tutorial:

https://www.youtube.com/watch?v=50qSBNTL5BM

In brief:

  1. To remove a clip:

    - [View] + Show Clips
    - Position clip cursor
    - [Edit] Split Clip
    - Select the clip and press Delete on keyboard
    
  2. To add another view to the end:

    - [Edit] Add clip to end
    - select the video file to add
    - treat as any other clip
    

Picture in Picture recording

Tutorial:

https://www.youtube.com/watch?v=mLyPrblmPp4

  Homebrew Cheatsheet

Homebrew cheatsheet

Commands

brew install git Install a package
brew uninstall git Remove/Uninstall a package
brew upgrade git Upgrade a package
brew unlink git Unlink
brew link git Link
brew switch git 2.5.0 Change versions
brew list --versions git See what versions you have

More package commands

brew info git List versions, caveats, etc
brew cleanup git Remove old versions
brew edit git Edit this formula
brew cat git Print this formula
brew home git Open homepage
brew search git Search for formulas

Global commands

brew update Update brew and cask
brew list List installed
brew outdated What’s due for upgrades?
brew doctor Diagnose brew issues

Brew Cask commands

brew cask install firefox Install the Firefox browser
brew cask list List installed applications

(*) Cask commands are used for interacting with graphical applications.

Credits:

https://devhints.io/homebrew

See also:

Uninstalling homebrew

From https://osxdaily.com/2018/08/12/how-uninstall-homebrew-mac/:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"

From https://apple.stackexchange.com/questions/82807/how-to-cleanly-remove-homebrew#82808:

cd /tmp
cd `brew --prefix`
rm -rf Cellar
brew prune
rm `git ls-files`
rm -r Library/Homebrew Library/Aliases Library/Formula Library/Contributions
rm -rf .git
rm -rf ~/Library/Caches/Homebrew

Homebrew Multi User Setup

From https://medium.com/@leifhanack/homebrew-multi-user-setup-e10cb5849d59:

  • Create a group brew and add Stan
  • brew doctor (Allen checks if he installed brew correctly)
  • sudo chgrp -R brew $(brew --prefix)/* (Change the group of homebrew installation directory)
  • sudo chmod -R g+w $(brew --prefix)/* (Allow group members to write inside this directory)
  • brew doctor (Stan checks if he can use homebrew)
  • We are done!

  Howto annotate a QuerySet with calculated properties and use them in ModelAdmin for filtering and sorting

Objectives

  • add a calculated property to a Model that can be shown in the admin list_display as a sortable and filterable column
  • have the new property available outside the admin
  • avoid any code duplications

the Manager

This can be achieved in a couple of ways:

  • using a custom manager with a method to annotate the queryset, or
  • delegate this activity to a custom QuerySet

In both cases, we can optionally override get_queryset() in the manager to have the calculated property available everywhere:

  • avantages: the calculated property is transparently added to any queryset; no further action is required
  • disadvantages: the computational overhead occurs even when the calculated properties are not used

In the following example, we opted to:

  • not override models.Manager.get_queryset()
  • have the computation available as a separate with_progess() method upon explicit request
  • use a custom QuerySet, so the new method can be chained with other elaborations in an arbitrary order
import datetime
from django.db import models
from django.db.models import Sum, Case, Q, F, When, Value as V, BooleanField, IntegerField
from django.db.models.functions import TruncDay, Now

class SomeQuerySet(models.QuerySet):

    def with_progress(self):
        return (self
            .annotate(
                _ongoing=Case(
                    When(
                        (Q(start_date=None) | Q(start_date__lte=TruncDay(Now()))) &
                        (Q(end_date=None) | Q(end_date__gte=TruncDay(Now()))) ,
                        then=V('True')
                    ),
                    default=(
                        V(False)
                    ),
                    output_field=BooleanField()
                ),
                _worked_hours=Sum("tasks__hours"),
                _progress=Case(
                    When(
                        hours_budget=0,
                        then=0
                    ),
                    default=(
                        F('_worked_hours') * 100.0 / F('hours_budget')
                    ),
                    output_field=IntegerField()
                ),
            )
        )


class SomeManager(models.Manager):

    def get_queryset(self):
        return SomeQuerySet(
            model=self.model,
            using=self._db,
            hints=self._hints
        )

the Model

In the model, we wrap every calculated value with a specific property, and add admin_order_field attribute to enable sorting.

from django.db import models
from .managers import SomeManager


class SomeModel(models.Model):

    objects = SomeManager()

    ...

    #@property
    def worked_hours(self):
        # Requires SomeModel.objects.with_progress()
        return self._worked_hours
    worked_hours.admin_order_field = '_worked_hours'
    worked_hours = property(worked_hours)

    #@property
    def progress(self):
        # Requires SomeModel.objects.with_progress()
        return self._progress
    progress.admin_order_field = '_progress'
    progress = property(progress)

    #@property
    def ongoing(self):
        # Requires SomeModel.objects.with_progress()
        return self._ongoing
    ongoing.boolean = True
    ongoing.admin_order_field = '_ongoing'
    ongoing = property(ongoing)

the ModelAdmin

In the ModelAdmin, the new properties can be added to list_display to obtain sortable columns in the changelist, and/or to readonly_fields to show them up in the changeview.

Here, we also override get_queryset() to make sure that the with_progress() is called as required.

If we did override SomeManager.get_queryset() instead, this wouldn't be necessary.

from django.contrib import admin
from .models import SomeModel

@admin.register(SomeModel)
class SomeModelAdmin(admin.ModelAdmin):

    list_display = [
        ...
        'ongoing',
        'worked_hours',
        'progress',
        ...
    ]
    readonly_fields = [
        ...
        'ongoing',
        'worked_hours',
        'progress',
        ...
    ]

    def get_queryset(self, request):
        queryset = (super().get_queryset(request)
            .with_progress()
            .select_related(
                ...
            )
        )
        return queryset

Filtering

To add filtering on a calculated field, we need one more step: subclass SimpleListFilter as shown below

class IsOngoingFilter(admin.SimpleListFilter):
    title = 'Ongoing'
    parameter_name = 'dummy'

    def lookups(self, request, model_admin):
        return (
            ('Yes', _('Yes')),
            ('No', _('No')),
        )

    def queryset(self, request, queryset):
        value = self.value()
        if value == 'Yes':
            return queryset.filter(_ongoing=True)
        elif value == 'No':
            return queryset.exclude(_ongoing=True)
        return queryset

then, in SomeModelAdmin:

@admin.register(SomeModel)
class SomeModelAdmin(BaseModelAdmin):

    list_filter = [
        ...
        IsOngoingFilter,
        ...
    ]

  Enable/Disable buttons based on HTML page status

<body class="dirty ..." >

    <style>
        a.button-save, a.button-cancel { opacity: 0.5; pointer-events: none; }
        body.dirty a.button-save, body.dirty a.button-cancel { opacity: 1.0; pointer-events: auto; }
        ...
    </style>

    ...

    <div id="toolbar1" class="toolbar">
        <a href="#" class="button-save btn btn-success">Salva</a>
        <a href="#" class="button-cancel btn btn-default">Annulla</a>
    </div>

To enable buttons:

$('body').addClass('dirty');

To disable buttons:

$('body').removeClass('dirty');

  Understand Group by in Django with SQL

Since I never really managed to think about aggregations in term of ORM rather than SQL, I'm practicing and summarizing here the examples described in the interesting article Understand Group by in Django with SQL by Haki Benita, where QuerySets and SQL are put side by side.

How to Use Aggregate Function

sql:

SELECT
    COUNT(id) as total
FROM
    auth_user;

python:

from django.db.models import Count

queryset = (User.objects
    .aggregate(
        total=Count('id'),
    )
)

The aggregate() function accepts an expression to count.

In this case, we used the name of the primary key column id to count all the rows in the table.

The result of aggregate is a dict, and the name of the argument to aggregate becomes the name of the key:

{"total": 891}

Annotations

Annotates each object in the QuerySet with the provided list of query expressions.

An expression may be

  • a simple value
  • a reference to a field on the model (or any related models)
  • or an aggregate expression (averages, sums, etc.) that has been computed over the objects that are related to the objects in the QuerySet.

Each argument to annotate() is an annotation that will be added to each object in the QuerySet that is returned.

See:

https://docs.djangoproject.com/en/2.0/ref/models/querysets/#annotate

How to Group By

We usually want to apply the aggregation on groups of rows:

sql:

SELECT
    is_active,
    COUNT(id) AS total
FROM
    auth_user
GROUP BY
    is_active

python:

queryset = (User.objects
    .values(
        'is_active',
    )
    .annotate(
        total=Count('id'),
    )
    .order_by()
)

To produce a GROUP BY we use a combination of values and annotate:

- values('is_active'): what to group by
- annotate(total=Count('id')): what to aggregate

The order is important: failing to call values before annotate will not produce aggregate results.

Interaction with default ordering or order_by()

When not requiring a specific ordering, we always add an empty ordering list (.order_by()) to prevent possible interactions from a model’s Meta.ordering

Fields that are mentioned in the order_by() part of a queryset (or which are
used in the default ordering on a model) are used when selecting the output
data, even if they are not otherwise specified in the values() call.

These extra fields are used to group “like” results together and they can make
otherwise identical result rows appear to be separate.

This shows up, particularly, when counting things.


Notes:

- Deprecated since version 2.2
- Starting in Django 3.1, the ordering from a model’s Meta.ordering won’t be
  used in GROUP BY queries, such as .annotate().values()
- Since Django 2.2, these queries issue a deprecation warning indicating to
  add an explicit order_by() to the queryset to silence the warning

See:

  https://docs.djangoproject.com/en/3.0/topics/db/aggregation/#interaction-with-default-ordering-or-order-by

How to Filter and Sort a QuerySet With Group By

You can use filter() ans order_by() anywhere in the query:

sql:

SELECT
    is_active,
    COUNT(id) AS total
FROM
    auth_user
WHERE
    is_staff = True
GROUP BY
    is_active
ORDER BY
    is_active,
    total

python:

queryset = (User.objects
    .values(
        'is_active',
    )
    .filter(
        is_staff=True,
    )
    .annotate(
        total=Count('id'),
    )
    .order_by(
        'is_staff',
        'total',
    )
)

Notice that you can sort by both the GROUP BY key and the aggregate field.

How to Combine Multiple Aggregations

To produce multiple aggregations on the same group, add multiple annotations:

sql:

SELECT
    is_active,
    COUNT(id) AS total,
    MAX(date_joined) AS last_joined
FROM
    auth_user
GROUP BY
    is_active

python:

from django.db.models import Max

queryset = (User.objects
    .values(
        'is_active',
    )
    .annotate(
        total=Count('id'),
        last_joined=Max('date_joined'),
    )
    .order_by()
)

How to Group by Multiple Fields

To group by multiple fields, list these fields in values()

sql:

SELECT
    is_active,
    is_staff,
    COUNT(id) AS total
FROM
    auth_user
GROUP BY
    is_active,
    is_staff

python:

queryset = (User.objects
    .values(
        'is_active',
        'is_staff',
    )
    .annotate(
        total=Count('id'),
    )
    .order_by()
)

  Make sure request.is_ajax() return True when sending a POST request from javascript

Looks like is_ajax just checks the HTTP_X_REQUESTED_WITH (looks like X-Requested-With in HTTP) header.

If it equals XMLHttpRequest, we have an ajax query.

Example:

promise = $.ajax({
    type: 'POST',
    url: url,
    data: data,
    cache: false,
    crossDomain: true,
    dataType: 'json',
    headers: {
        'X-CSRFToken': getCookie('csrftoken'),
        'X-Requested-With': 'XMLHttpRequest'  // make sure request.is_ajax() return True on the server
    }
});

Credits: https://stackoverflow.com/questions/8587693/django-request-is-ajax-returning-false/27240729#27240729

  Google maps alternatives

Buonasera a tutti, con il mio team stiamo provando vari servizi di geocoding (reverse/forward) al di là del classico Google. Tra i migliori che abbiamo trovato vorrei segnalare:

Ad ora il servizio che si è rilevato più preciso tra quelli descritti è Here, molto più preciso anche di Photon e Mapbox combinati insieme (il tutto per lavorare con pacchi di CSV di indirizzi da parsare scritti nel peggiore dei modi). Inoltre la versione free ha un limite di richieste molto ampio. Voi cosa usate? Consigli in merito?

Andrea Stagi

Luca Adamo Andrea io uso HERE Maps in un progetto in produzione con circa 9-11M chiamate mese, rispetto a Google ho perso un po' sul geocoding del POI extra UE (attività commerciali strane magari o relativamente nuove) ma risparmio il 90% in fattura mensile (da 15K USD mese Google spendiamo un decimo). In più se fai routing o integri sistemi di previsioni ETA o guida turn-to-turn hai un sacco di opzioni per gestire mezzi non convenzionali (es. truck, che non può girare se la curva è oltre X gradi) e modelli di costo della route custom (in pratica il peso degli archi del grafo te lo puoi decidere in maniera più fine in funzione del mezzo che hai). Secondo me complessivamente è già meglio delle API Google e ci sto passando altri progetti di dimensione analoga. MapBox provato per "gioco" e per farmi un tile-server mio (ottimo per quello con Openlayer), non come servizi REST per cui non so. L'altro lo scopro adesso :)

Andrea Stagi Andrea Stagi Veramente top! Grazie per la panoramica del servizio, soprattutto lato costi diventa davvero interessante

  How to enable Screen Sharing on Macs via Terminal

sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -access -off -restart -agent -privs -all -allowAccessFor -allUsers

Credits:

https://www.techrepublic.com/article/how-to-enable-screen-sharing-on-macs-via-terminal/

  Easy Printing

row = [100, "android", "ios", "blackberry"]
print(', '.join(str(x) for x in row))
print(*row, sep=', ')

Credits:

https://medium.com/better-programming/python-shortcut-tricks-18d676eb58ce

  Redirect to an arbitrary URL After Saving in Django Admin

Put following change_view-method in your admin.py and now you can make links to admin change page that return you back to where you came from:

class BlogEntryAdmin(admin.ModelAdmin):

    ...

    def change_view(self, request, object_id, form_url='', extra_context=None):
        result = super(BlogEntryAdmin, self).change_view(request, object_id, form_url, extra_context)
        if request.GET.get('return_to', False):
            result['Location'] = request.GET['return_to']
        return result

In your template you'd have something like:

{% if request.user.is_staff %}
    <a href="/admin/blog/blogentry/{{ entry.pk }}/?return_to={{ entry.get_absolute_url }}">Edit in admin</a>
{% endif %}