Building a Flask Foundation

Resources

Not exactly how we do it, but there are additional resources here:
https://www.digitalocean.com/community/tutorials/how-to-deploy-a-flask-application-on-an-ubuntu-vps

Flask Documentation
https://flask.palletsprojects.com/en/1.1.x/

Great book. While it is not free, you can find it in Humble Bundles for a great deal if you watch.
https://www.amazon.com/dp/B07B8DCCN7/ref=cm_sw_em_r_mt_dp_U_ocMGEb8X434MT

Instructions

At this point, you should already have a LAMP stack set up. If you don’t, you can follow my guide here: LAMP Stack on Ubuntu Server

Now that we have our LAMP stack as a foundation, we are going to start building our application layer. Our application layer is going to utilize Flask for the web framework. Flask is Python based and really modular, so it works well for projects of all sizes. It also has a lot of support and libraries which makes the development process much faster.

Flask Directory

To start, we are going to build our Flask directory structure.

Navigate to the apache web folder
cd /var/www
Create a new directory for our project
sudo mkdir app
Assign your user as the owner, and make www-data the group (so it has web access) *Be sure to change my username to yours.
sudo chown tannercrook:www-data app
Navigate into our project’s directory
cd app

We are going to use this as our base hub. This is an area that can hold files that will be separate from the publicly available resources. For example, we may put a database connection file here containing our database credentials for the app to use. Obviously, we don’t want the internet to have access to this file.

So, let’s build our Flask folder and structure:

Build the flask app directory and move into it
mkdir app
cd app
Build the various directories for Flask
mkdir -p models static/styles templates
Build our main app python file
touch app.py

We should now have our base structure for Flask:
/var/www/app/app
├── app.py
├── models
├── static
│   └── styles
└── templates

 

Our Flask Application

Now we need to set up our app.py file so we can configure and test Apache.

* We have already generated a secret key, but if you want to generate one of your own (if you host your own) you can generate it by using the instructions here.


from flask import Flask 
from flask import render_template

app = Flask(__name__)
app.config['SECRET_KEY'] = '\xd2\x04S4\xbc\xce\xe2\x17\xfb\xff\x19C@\xa6e\xc2\xf4\x18\xad\xe8\xc4\xcb'


@app.route('/')
def index():
    return 'Hello, World!'

Place the following code of your empty app.py file using either FTP and your editor or the command line (editing with nano):
nano app.py
To paste: ctrl+shift+v
To save: ctrl+o

Once that is saved, we need to set up our python environment.
python3 -m venv venv
This will create a container for python so we can install components that won’t interfere with other parts of the system. Or if you wanted to have multiple Flask apps running, you don’t have to worry about interference between the two.

To activate our environment
source venv/bin/activate
And you should now see (venv) at the beginning of your prompt.

Now we can install the python dependencies we used in our app.py file.
pip3 install wheel
pip3 install flask

Once we have done that, our Flask directory is set up!

 

Apache and WSGI Configuration

For our webserver, we are using Apache which will direct to WSGI to serve flask URL.

First, we need to set up our WSGI config file in a main directory
cd ..
pwd
Should return /var/www/app

Let’s create the file:
touch app.wsgi
nano app.wsgi


#!/usr/bin/python
import sys
import logging
logging.basicConfig(stream=sys.stderr)
sys.path.insert(0,"/var/www/app/app")

from app import app as application
application.secret_key = '\xd2\x04S4\xbc\xce\xe2\x17\xfb\xff\x19C@\xa6e\xc2\xf4\x18\xad\xe8\xc4\xcb'

WSGI is now ready to go! Now we just need to configure Apache to point to our Flask configuration.

Change permissions on our files
sudo chown -R tannercrook:www-data app
sudo chmod -R 774 /var/www/app
Let’s install the Apache WSGI mod
sudo apt install libapache2-mod-wsgi-py3
Ensure it is enabled
sudo a2enmod wsgi

We will replace the existing default website config with our new config.
sudo nano /etc/apache2/sites-available/000-default.conf

 


<VirtualHost *:80>
        ServerAdmin webmaster@local

        # Build WSGI for Flask
        WSGIDaemonProcess app python-home=/var/www/app/app/venv
        WSGIScriptAlias / /var/www/app/app.wsgi process-group=app application-group=%{GLOBAL}

        <Directory /var/www/app/app/>
                Order allow,deny
                Allow from all
        </Directory>
        
        Alias /static /var/www/app/app/static
        <Directory /var/www/app/app/static/>
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        LogLevel warn

        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>



 

Restart Apache
sudo service apache2 restart

If all went well, you should now be able to enter the IP Address of your server in a browser and see ‘Hello, World!’. This means that you have a working LAMP stack with Flask!

 

 

 

 

 

LAMP Stack on Ubuntu Server

Resources

The Linux Command Line by William Shotts
Free Ebook: http://www.linuxcommand.org/tlcl.php/

Instructions

At this point you should already have a functioning Ubuntu Server up and running. Now we will configure it to have a LAMP stack!

Our LAMP stack will consist of the following:

  • Linux (Ubuntu Server 18.04)
  • Apache
  • PostgreSQL (instead of MySQL)
  • Python (Flask)

The Linux step of LAMP

The first thing we want to do is verify that our server is up-to-date. Note: When prompted for a password in the terminal, it will not show typing, but it is working. Simply type your password and hit enter.
sudo apt update will make sure our repository is up to date so we pull from the correct locations.
sudo apt upgrade will upgrade the software on our server to the newest versions.

Once that finishes, we are all set for the Linux step!

Installing Apache

Apache is software that allows us to run a webserver to deliver files via http(s). In more basic terms, it is a piece of software that will allow us to host our own website, albeit for now on our own network only.

The beautiful part about Linux is how easy it is to install software. Many people shy away from using a terminal because initial responses are usually that it is complex; however, you will find that it is simple and the quickest method.

To install Apache simply execute the command:
sudo apt install apache2

We can verify that everything is working correctly by testing the default webpage. First, we need to find the ip address of our server.

Find the IP address of your server:
ip address

The results will include 2 adapters, the first will be 127.0.0.1 for your internal loopback address. The second is the IP address of your server and should be an IP address in your own network like 192.168.0.X or 10.0.0.X. Take note of the address.

To test our webserver, simply put that IP address into the address bar of the browser of your choice. If you get the following webpage, you are working!

Apache Default Page

 

Installing PostgreSQL

PostgreSQL will be the database we use for our stack. I prefer PostgreSQL as it is a free, open-source product that has all of the features of the most advanced database systems. It is growing in popular among developers and businesses.

To install PostgreSQL
sudo apt install postgresql postgresql-contrib

We will need to do more to setup and configure Postgre for our project. However, for now we just needed it installed. So we are all set for the database portion of our LAMP stack!

 

Installing Python

The last part of our LAMP stack will be Python. Ubuntu Server 18.04+ should have Python3 already installed.

To verify that it is installed
sudo apt install python3

We also need a few extra python components.
sudo apt install python3-venv
sudo apt install python3-pip

And just like that we have our Python components ready.

 

The LAMP Stack Base

Our LAMP stack is all installed and ready but there still is quite a bit of configuring to do.  This is a good base though depending upon which technologies you are using for your project. In our case, we will finish configuring our components as we start building and tying the system together.

 

Bonus: FTP

If  you want FTP on your server, so you can edit files on your host machine with an editor of your choice, you can install it pretty easily.

Install vsftp
sudo apt install vsftpd

Configure it so it is write enabled
sudo nano /etc/vsftpd.conf
Uncomment the line:

write_enable=YES

Restart the service:
sudo service vsftpd restart

Then you  should be able to connect to your server using your credentials using the FTP tool of your choice (Like Filezilla).

 

 

 

 

 

 

 

 

 

Ubuntu Server as Virtual Machine

Resources

VirtualBox (https://www.virtualbox.org/wiki/Downloads) or you can also use VMWare Player or VMWare Workstation or Fusion if you have one of those.

Download the latest LTS Ubuntu Server (20.04): https://releases.ubuntu.com/20.04.1/ubuntu-20.04.1-live-server-amd64.iso

Instructions

Option 1: VMware Player

  1. Download the Ubuntu Server ISO (listed at top of this page).
  2. Select Create a new virtual machine
  3. Select Use ISO image and browse and select the .iso you downloaded. Press ‘Next’.
  4. Enter your name, username, and password. Select ‘Next’.
  5. Choose a location to save your VM files. Select ‘Next’.
  6. Leave storage size at least 20GB. Choose option to store as single file. Select ‘Next’.
  7. Choose ‘Customize Hardware’.
    1. Select ‘Memory’ and set to 2048mb (2GB).
    2. Select ‘Network Adapter’ and select radial ‘Bridged’.
    3. Select ‘Close’.
  8. Select ‘Finish’.

You can now skip to the Ubuntu Installation section.

Option 2: VirtualBox

  1. Install VirtualBox and Download the Ubuntu Server ISO.
  2. In VirtualBox, select Machine > New from the top menu to build a new Virtual Machine:
    1. Screenshot from 2020-03-30 09-42-18
    2. On the next screen, designate either 1024mb or 2048mb of RAM for the server. (Preferably 2GB but if your computer has low RAM, use 1GB)
    3. On the next screen, select to ‘Create a virtual hard disk now’ and then ‘Virtualbox Disk Image’
    4. On the next screen, select ‘Dynamically allocated’
    5. On the next screen, set the disk size to 15GB.  You can set more if you would like, but it is very lean.
  3. We have the container for our virtual machine made. Now we need to put the installation media into our VM so we can boot from it and install.
    1. In the VirtualBox Manager window, select your newly created VM and select the Settings button (or you can right click it and select Settings)
    2. In the resulting screen, select the Storage tab on the left.
    3. Select the optical drive under Controller: IDE
    4. In the Attributes section, click the CD image next to the drop down menu, and use the option to Choose a file to browse to the downloaded Ubuntu Server ISO file.
    5. Check the box for ‘Live CD/DVD’
    6. The result should look like the following:
      1. Screenshot from 2020-03-30 10-11-25
  4. We also want to set up the Virtual Machine’s network so we can see it from our host computer. While still in the Settings window:
    1. Select the Network tab on the left.
    2. On the Adapter 1 tab, change the ‘Attached to: ‘ from NAT to Bridged Adapter
  5. Select the OK button to save our changes.

 

Installing Ubuntu Server

  1. In the VM Manager window, select the Server VM we created, and select Start.
  2. Install Ubuntu Server using the installation guide *Note that the terminal is text-only input. You cannot use a mouse and will use the arrow keys to navigate. Press enter to select. *
    1. Select your language
    2. Select ‘Update to the new installer’
    3. Select your keyboard configuration
    4. On the Network Connections page, select Done to accept defaults
    5. On the Configure Proxy page, select Done to skip
    6. On the Configure Ubuntu Archive Mirror page, select Done to accept defaults
    7. On the Guided Storage Configuration page, ensure Use an entire disk is select, then arrow down and select Done
    8. On the Storage Configuration, select Done to confirm and then select Continue to confirm again.
    9. On the Profile setup screen, enter your information and select Done:
      1. Screenshot from 2020-03-30 10-36-10
    10. On the SSH Setup page, check the box for Install OpenSSH server and select Done.
    11. On the Featured Server Snaps, select Done to skip.
    12. At this point, the operating system will install and update. Wait until it finishes and the option will say ‘Reboot’.
  3. After selecting ‘Reboot’, it will begin to reboot but will show ‘Please remove the installation medium’.
    1. In the VirtualBox Manager window (the one that lists all your virtual machines) Right click the VM and select Close > Power off. Confirm to Power off.
    2. Select the Settings button.
    3. Select Storage in the left panel.
    4. Under the Controller: IDE select the ubuntu-18.04 cd.
    5. Select the cd image next to the drop down menu and select ‘Remove disk…’
    6. Select OK
  4. Start the Virtual Machine again.
    1. It will do some initial setup, press the return key once it pauses to access the login prompt
      1. Screenshot from 2020-03-30 10-47-02
    2. Enter your username
    3. Enter your password * Note it will not show any indication of typing for security purposes. Type the password and hit enter. *
    4. Once your logged in, you should see a screen like the following:
      1. Screenshot from 2020-03-30 10-49-03

 

Congratulations! You have a Ubuntu Server virtual machine running!

tryManjaroHeader

Let’s Try Manjaro

We have all been there, booting up our trusty distro again, knowing exactly what our workflows are and will be. Ready to get things done. When suddenly, a shot of curiosity pulses through your neurons and you think, ‘What if this #insert-distro-here I have been hearing about is as great as everyone says?’.

This was me thinking about Arch based Manjaro.

It seemed like Manjaro was popping up everywhere from articles, podcasts, forums, and chatter in the office. Having explored their website, I was immediately intrigued because it was one of the best distribution websites I had seen. It added a marketing element that seems to be lacking in the Linux realm and the method is refreshing in a familiar way and provides an avenue to present the best features of the product in an exciting way. The result is that everyone with a spark of curiosity will end up with a bootable USB. The excitement even starts when deciding which download button to click.

Manjaro WallpaperManjaro XFCE Download Listing

Linux is often pegged as being too technical. I imagine that people unfamiliar immediately picture hackers, text-only websites, and anime upon hearing the term. This confusion can not always be dismissed as new users are often faced with a trundle of choices just to get started: Which distribution do I pick? Which desktop environment do I want? What is a window manager? Which anime should I use as my wallpaper?

Manjaro guides users both old an new with a beautiful download page that features each of their pre-made variations (XFCE, Gnome, and KDE). Each of the listings includes a fabulous video demonstrating various components, which allows users to quickly identify which look and feel they should go with. Linux plebs and pros alike can appreciate the delivery method.

I had been using Ubuntu and variations like Pop!_OS as my daily drivers for years, and when I read that XFCE was the front-man version of Manjaro I knew I had to give it a shot. To avoid disrupting my work entirely, I decided to install on my laptop: An XPS13 Developer Edition that I use between my work and home desktop and during meetings (~4 hours a day). It would be enough to really put the rubber to the road without risking being unproductive. Using Pop!_OS’s Popsickle USB creator, I made the USB with ease and was installed and booted-up in no-time. The installer was very straightforward and modern and would work for any level of user.

Having used Gnome exclusively for some time, XFCE was definitely new. It was snappy, fast, and extremely customizable. After using it for some time, I found that the simplicity came with some downsides. On my laptop’s 4k display, some of the icons and artifacts were not properly scaled. The cursor wouldn’t scale in some of my foundation pieces of software like Firefox. It was a little discouraging after reading so much about XFCE and knowing that it was Manjaro’s front-runner. Knowing Linux well I knew that there was a way to fix it, but I wanted to jump in and tweak things, not fix them. I had a installation of Manjaro Gnome shortly thereafter.

Manjaro GnomeManjaro Gnome

Using Manjaro Gnome better allowed me to see the difference between the Arch-based Manjaro and the debian based distributions I traditionally use. While I plan on revisiting XFCE in the future, Gnome ended up being an easy option to stick with both for the familiarity and functionality.

Rolling Releases

The biggest difference that caught my attention was the rolling releases on Manjaro. Because it is based on Arch, software does not need to be added to the channel you are on to upgrade. Running sudo apt up… I mean sudo pacman -Syu will upgrade the various components if they were upgraded at the source. This means that you get the newest packages, as soon as they are available. My XPS13 Developer Edition running Ubuntu 19.10 always seemed to have some driver related issue (this was experienced by 4 other co-workers as well) such as wifi disconnecting by itself on enterprise networks, the computer not hibernating properly when the lid was closed, and waking up while the lid was closed. These issues could be avoided on Ubuntu by manually adding the PPA for the driver and installing it. But the issue had already been fixed, it just wasn’t in the release channel Ubuntu was using.

Manjaro Gnome

Also, while on Ubuntu, wayland on my hidpi display had enough issues to make it nuisance enough not to use. On Manjaro, wayland works flawlessly and I have been using it as my window manager since the switch.

With the rolling releases of Manjaro, they changes were already there. New fixes will continually always be there as soon as they are ready without having to wait for packages to be added to the channel or doing anything manual.

Software Availability

Manjaro surprised me a little by having support for snap and flatpak right out of the box. Between those options and the repositories, almost all the software I used on a daily basis was easily found and installable. You noticed how I said almost…

In my day job I am a database administrator and one of my main systems uses Microsoft SQL Server. I was tickled when I saw that Microsoft gave people like me a Christmas present and had released Azure Data Studio for Linux at the end of 2019. While I have and use an open source application called DBeaver for all of my database work, Azure Data Studio had a lot of nice SQL Server support and features that really helped. However, Microsoft only packages the tool in deb and rpm formats. Leaving everyone else to compile themselves. We also had just implemented Microsoft Teams, and Microsoft’s applauded efforts to bring Teams to Linux was met with my heartache as it too was only available in deb and rpm formats with no compile yourself option.

This may or may not be an issue for many people, as it is a small portion of software that is unavailable. Additionally, there are work arounds for many, as in my case I use the web for Teams and use DBeaver on my laptop. My hope is that the companies that are just starting to provide their software on Linux, package using flatpak or snap so they are more widely available.

Overall Feel and Choice

Manjaro Gnome offers a modern and sleek feel that will be familiar and easy to use for everyone. Yet somehow, manages to still add its own unique flavor that makes it exciting and fun to use. Things like Manjaro Hello allow you to instantly change the layout of gnome to best match your style that includes options like a modern macOS-like layout, a windows-like traditional layout, and many more that explore new and unique ways to live in your computer. On top of the layout, a wide variety of settings can be tweaked to put the finishing touches on.

Manjaro Gnome Layouts

I can’t help but feel that Manjaro really opened the door to me to experience what Linux Desktop could be. Don’t get me wrong, the other distributions I have used and mentioned are fantastic and I still use them daily. But out-of-the-box they seem to try to pick their best combination and offer their opinion to you, which in most cases, is pretty good. However, Manjaro uses Arch as a fridge full of ingredients and is the gourmet chef willing to make whatever you ask it to, and all you had to do was ask. Manjaro is the freedom of choice starting at the download page and ending only when you smugly close the lid to your laptop. It was an amazing feeling to know that I could choose different combinations of components without comprising functionality.

Ultimately, choice is what makes Linux desktop so rich, vibrant, and fulfilling. I use various distributions daily in different capacities and like my children, love them each for unique character traits and put them into time-out for others. There may never be a perfect distribution for everyone, but there may be a perfect distribution for you. Manjaro is a distribution that gives you the opportunity to make your own perfect dish, without hindering your creativity. The most important part: a person of any skill set can do it, whether you have an anime wallpaper or not.

Manjaro DesktopOne Punch Man Wallpaper

Visit or get Manjaro

Infinite Campus: Get Special Ed Counts by Year, School, and Snapshot Date

I was tasked with getting a count of special education students historically. As always, I wanted a solution that was elegant, future-proof, and easily manipulated. The completed code is at the bottom of the post with an explanation between!

Let’s break it down

We begin by gathering the foundational data of the query: the school year, the school name, and the enrollments which we will use to count or aggregate later.

SELECT DISTINCT
  cal.endYear 
, sch.name 
FROM Calendar cal
INNER JOIN School sch 
ON cal.schoolID = sch.schoolID
INNER JOIN Enrollment e 
ON cal.calendarID = e.calendarID
WHERE e.specialEdStatus = 'Y'
ORDER BY 1,2;

We are selecting with the DISTINCT clause and we don’t have any enrollment data in the SELECT list which means that we will see the result we will end up aggregating to. Just know that the enrollment data exists behind the scenes, we just aren’t selecting it.

Now, we want to make it so each row in our existing result set, has a record for each snapshot date. We also want to do this so we don’t have to put string literal dates for every snapshot. My solution is a CROSS JOIN to a query that builds a list of every calendar year’s set of snapshot dates. We also include the calendar year for each record to match on later. The query can be ran standalone to see the result set.

SELECT CONCAT((cal.endYear-1),'-10-15') AS snapDate, cal.endYear FROM Calendar cal
UNION 
SELECT CONCAT(cal.endYear,'-03-15') AS snapDate, cal.endYear FROM Calendar cal
UNION 
SELECT CONCAT(cal.endYear,'-05-15') AS snapDate, cal.endYear FROM Calendar cal;

Now that we have our snapshot dates, we need to CROSS JOIN it with our previous result set.

CROSS JOIN (
    SELECT CONCAT((cal.endYear-1),'-10-15') AS snapDate, cal.endYear FROM Calendar cal
    UNION 
    SELECT CONCAT(cal.endYear,'-03-15') AS snapDate, cal.endYear FROM Calendar cal
    UNION 
    SELECT CONCAT(cal.endYear,'-05-15') AS snapDate, cal.endYear FROM Calendar cal
) snap

Since this will generate every possible combination (Cartesian Product or Table A x Table B) of the result of the CROSS JOIN, we need to filter out the non-matching records. So we add to our WHERE clause WHERE cal.endYear = snap.endYear. We also add our new snapshot date to our select list: , snap.snapDate.

We add a line to check that the student’s enrollment record is active on the snapshot date. AND snap.snapDate BETWEEN e.startDate AND ISNULL(e.endDate, CONCAT(cal.endYear,'-05-15'))

Finally, we finish it off by aggregating the count of students that are marked as Special Education for each record by adding the aggregate function to our select list:
, COUNT(DISTINCT e.personID) AS snapCount

and a GROUP BY clause to designate how we want to aggregate (in our case, by the year, school, and date):
GROUP BY cal.endYear, sch.name, snap.snapDate

The final product

You can find the completed query below. Thanks for reading and I hope this helps!