CompleteNoobs Local Wiki In Docker

Starting Environment

• Hardware: HP EliteDesk 800 G1
• Operating System: Ubuntu-Mate 24.04
• Installation Type: Fresh Install
• Initial Setup Post-Install: Package Update: sudo apt update && sudo apt upgrade -y
• Additional Software: Installed just to record screen for this tut and not needed sudo apt install simplescreenrecorder

Docker Installation Guide

Load video

YouTube

YouTube might collect personal data. Privacy Policy

ContinueDismiss

Preparation

Before we begin, make sure you're logged in with a user account that has sudo privileges.

Update System Packages

Update your package list to ensure you have the latest versions of packages:

sudo apt update

Install Docker Prerequisites

Install the necessary packages for Docker setup:

sudo apt install -y apt-transport-https ca-certificates curl software-properties-common

Setup Docker Repository

• Add Docker's Official GPG Key:

  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

• Add the Docker Repository:

  echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker and Docker Compose

• Update Package List Again:

  sudo apt update

• Install Docker Engine, CLI, Containerd, and Additional Tools:

  sudo apt install -y docker-ce docker-ce-cli containerd.io python3-bs4 python3-requests docker-compose

• Install Docker Compose:

Here we're downloading the latest version of Docker Compose:

  sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

Make the Docker Compose binary executable:

  sudo chmod +x /usr/local/bin/docker-compose

Verify Installation

Check if Docker and Docker Compose are installed correctly:

docker --version
docker-compose --version

Configure User Permissions

To run Docker commands without sudo, add your user to the docker group:

sudo usermod -aG docker $USER

Note: After adding your user to the docker group, you'll need to log out and log back in for the changes to take effect. If you do not log out and back in, Or you do not add your $USER to the docker group, you will be required to use sudo in some cases. such as ..

a way to apply group changes without logging out and back in - tip:

exec sudo su -l $USER

This command will replace your current shell with a new login shell for your user, which will have the updated group memberships. Both of these methods will apply the group changes immediately, allowing you to use LXD commands without having to log out and back in. Remember, these changes only apply to the current terminal session. If you open a new terminal window, you might need to run the command again or log out and back in for the changes to take effect system-wide.

Creating a Wiki and Importing an XML Dump

Setup the Wiki Environment

Create Directory for the Wiki

mkdir noobwiki
cd noobwiki

Create Docker Compose File

Edit your docker-compose.yml file:

• set editor

$EDITOR docker-compose.yml

Contents of docker-compose.yml:

services:
  database:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: wikipass
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
    volumes:
      - ./mysql_data:/var/lib/mysql

  mediawiki:
    image: mediawiki
    ports:
      - 8080:80
    links:
      - database
    volumes:
      - ./mediawiki_data:/var/www/html/images
#      - ./LocalSettings.php:/var/www/html/LocalSettings.php # This line is commented out initially

Note: The LocalSettings.php volume mapping is commented out to prevent automatic creation during initial setup.

Python Script for Setup and XML Import

Script Details

Create a Python script to automate the setup and import process:

$EDITOR setup_and_import.py

Newer Script that will list newest first - list from new to old in order:

• Passed early testing

import os
import time
import requests
from bs4 import BeautifulSoup
import re
import subprocess
import sys

BASE_URL = "https://xml.completenoobs.com/xmlDumps/"

def parse_date_from_dump(dump_name):
    """Parse DD_MM_YY format and return a sortable date tuple."""
    match = re.match(r'(\d{2})_(\d{2})_(\d{2})\.Noobs', dump_name)
    if match:
        day, month, year = match.groups()
        # Convert 2-digit year to 4-digit (assuming 00-49 = 2000-2049, 50-99 = 1950-1999)
        year_int = int(year)
        if year_int <= 49:
            full_year = 2000 + year_int
        else:
            full_year = 1900 + year_int
        return (full_year, int(month), int(day))
    return (0, 0, 0)  # fallback for unparseable names

def get_available_dumps():
    response = requests.get(BASE_URL)
    soup = BeautifulSoup(response.text, 'html.parser')
    dumps = []
    for link in soup.find_all('a'):
        href = link.get('href')
        if re.match(r'\d{2}_\d{2}_\d{2}\.Noobs/$', href):
            dumps.append(href.rstrip('/'))
    
    # Sort by actual date (newest first)
    return sorted(dumps, key=parse_date_from_dump, reverse=True)

def get_dump_files(dump):
    response = requests.get(BASE_URL + dump)
    soup = BeautifulSoup(response.text, 'html.parser')
    files = {}
    for link in soup.find_all('a'):
        href = link.get('href')
        if href.endswith('.xml'):
            # Extract file info (size, date) from the next sibling text
            file_info = ""
            next_sibling = link.next_sibling
            if next_sibling:
                file_info = next_sibling.strip()
            files[href] = file_info
    
    # Sort XML files by name (newest first, assuming filename contains date/time info)
    sorted_files = dict(sorted(files.items(), key=lambda x: x[0], reverse=True))
    return sorted_files

def download_file(url, filename):
    print(f"Downloading from: {url}")
    response = requests.get(url, stream=True)
    response.raise_for_status()
    
    total_size = int(response.headers.get('content-length', 0))
    downloaded = 0
    
    with open(filename, 'wb') as f:
        for chunk in response.iter_content(chunk_size=8192):
            if chunk:
                f.write(chunk)
                downloaded += len(chunk)
                if total_size > 0:
                    progress = (downloaded / total_size) * 100
                    print(f"\rProgress: {progress:.1f}%", end='', flush=True)
    print()  # New line after progress

def run_command(command):
    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
    output, error = process.communicate()
    return output.decode(), error.decode(), process.returncode

def check_docker_permissions():
    """Check if Docker is accessible and provide helpful error messages."""
    try:
        output, error, return_code = run_command("docker info")
        if return_code != 0:
            if "permission denied" in error.lower():
                print("ERROR: Docker permission denied!")
                print("Solutions:")
                print("1. Run this script with sudo: sudo python3 setup_and_import.py")
                print("2. Or add your user to docker group: sudo usermod -aG docker $USER")
                print("   Then log out and log back in.")
                return False
            else:
                print(f"Docker error: {error}")
                return False
        return True
    except Exception as e:
        print(f"Error checking Docker: {e}")
        return False

def setup_mediawiki():
    print("Checking Docker permissions...")
    if not check_docker_permissions():
        sys.exit(1)
    
    print("Setting up MediaWiki...")
    output, error, return_code = run_command("docker-compose up -d")
    if return_code != 0:
        print(f"Error starting Docker containers: {error}")
        sys.exit(1)
    
    time.sleep(10)  # Wait for services to start

    print("Please complete the MediaWiki installation by visiting http://localhost:8080")
    print("Use the following database settings:")
    print("Database host: database")
    print("Database name: my_wiki")
    print("Database username: wikiuser")
    print("Database password: wikipass")
    input("Press Enter when you have completed the installation and downloaded LocalSettings.php...")

    if not os.path.exists('LocalSettings.php'):
        print("LocalSettings.php not found. Please place it in the current directory.")
        sys.exit(1)

    output, error, return_code = run_command("docker-compose restart mediawiki")
    if return_code != 0:
        print(f"Error restarting MediaWiki: {error}")
        sys.exit(1)
    
    print("MediaWiki setup completed.")

def import_xml_to_mediawiki(xml_file):
    command = f"docker-compose exec -T mediawiki php maintenance/importDump.php < {xml_file}"
    print(f"Importing {xml_file} into MediaWiki...")
    output, error, return_code = run_command(command)
    if return_code != 0:
        print(f"Error importing XML: {error}")
        sys.exit(1)
    print("Import completed successfully.")

def main():
    # Check Docker permissions early
    if not os.path.exists('LocalSettings.php'):
        setup_mediawiki()
    else:
        # Even if LocalSettings.php exists, check Docker permissions for import
        print("Checking Docker permissions...")
        if not check_docker_permissions():
            sys.exit(1)

    print("Fetching available dumps...")
    dumps = get_available_dumps()
    if not dumps:
        print("No dumps found!")
        sys.exit(1)
    
    print("\nAvailable dumps (newest first):")
    for i, dump in enumerate(dumps):
        marker = " (NEWEST)" if i == 0 else ""
        print(f"{i + 1}. {dump}{marker}")
    
    # Default to newest dump (index 0)
    default_choice = 1
    choice_input = input(f"\nEnter the number of the dump you want to download (default: {default_choice} - newest): ").strip()
    
    if choice_input == "":
        choice = default_choice - 1
    else:
        try:
            choice = int(choice_input) - 1
            if choice < 0 or choice >= len(dumps):
                print("Invalid choice. Using newest dump.")
                choice = 0
        except ValueError:
            print("Invalid input. Using newest dump.")
            choice = 0
    
    selected_dump = dumps[choice]
    print(f"\nSelected dump: {selected_dump}")
    
    print("Fetching XML files from selected dump...")
    files = get_dump_files(selected_dump)
    
    if not files:
        print("No XML files found in the selected dump!")
        sys.exit(1)
    
    xml_files = list(files.keys())
    
    print(f"\nAvailable XML files in {selected_dump} (newest first):")
    for i, xml_file in enumerate(xml_files):
        file_info = files[xml_file]
        marker = " (RECOMMENDED)" if i == 0 else ""
        print(f"{i + 1}. {xml_file}{marker}")
        if file_info:
            print(f"    {file_info}")
    
    # Default to newest XML file (index 0)
    default_xml_choice = 1
    xml_choice_input = input(f"\nEnter the number of the XML file you want to download (default: {default_xml_choice} - newest): ").strip()
    
    if xml_choice_input == "":
        xml_choice = default_xml_choice - 1
    else:
        try:
            xml_choice = int(xml_choice_input) - 1
            if xml_choice < 0 or xml_choice >= len(xml_files):
                print("Invalid choice. Using newest XML file.")
                xml_choice = 0
        except ValueError:
            print("Invalid input. Using newest XML file.")
            xml_choice = 0
    
    selected_xml = xml_files[xml_choice]
    print(f"\nSelected XML file: {selected_xml}")
    
    # Check if file already exists
    if os.path.exists(selected_xml):
        overwrite = input(f"\n{selected_xml} already exists. Overwrite? (y/N): ").strip().lower()
        if overwrite not in ['y', 'yes']:
            print("Using existing file.")
        else:
            print(f"Downloading {selected_xml}...")
            download_file(BASE_URL + selected_dump + '/' + selected_xml, selected_xml)
    else:
        print(f"Downloading {selected_xml}...")
        download_file(BASE_URL + selected_dump + '/' + selected_xml, selected_xml)
    
    print("Importing XML dump into MediaWiki...")
    import_xml_to_mediawiki(selected_xml)
    print("\nImport completed successfully!")
    print(f"Your MediaWiki instance should now be available at: http://localhost:8080")

if __name__ == "__main__":
    main()

Contents of setup_and_import.py:

import os
import time
import requests
from bs4 import BeautifulSoup
import re
import subprocess
import sys

BASE_URL = "https://xml.completenoobs.com/xmlDumps/"

def get_available_dumps():
    response = requests.get(BASE_URL)
    soup = BeautifulSoup(response.text, 'html.parser')
    dumps = []
    for link in soup.find_all('a'):
        href = link.get('href')
        if re.match(r'\d{2}_\d{2}_\d{2}\.Noobs/$', href):
            dumps.append(href.rstrip('/'))
    return sorted(dumps, reverse=True)

def get_dump_files(dump):
    response = requests.get(BASE_URL + dump)
    soup = BeautifulSoup(response.text, 'html.parser')
    files = {}
    for link in soup.find_all('a'):
        href = link.get('href')
        if href.endswith('.xml'):
            files[href] = link.next_sibling.strip()
    return files

def download_file(url, filename):
    response = requests.get(url)
    with open(filename, 'wb') as f:
        f.write(response.content)

def run_command(command):
    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
    output, error = process.communicate()
    return output.decode(), error.decode(), process.returncode

def setup_mediawiki():
    print("Setting up MediaWiki...")
    os.system("docker-compose up -d")
    time.sleep(10)  # Wait for services to start

    print("Please complete the MediaWiki installation by visiting http://localhost:8080")
    print("Use the following database settings:")
    print("Database host: database")
    print("Database name: my_wiki")
    print("Database username: wikiuser")
    print("Database password: wikipass")
    input("Press Enter when you have completed the installation and downloaded LocalSettings.php...")

    if not os.path.exists('LocalSettings.php'):
        print("LocalSettings.php not found. Please place it in the current directory.")
        sys.exit(1)

    os.system("docker-compose restart mediawiki")
    print("MediaWiki setup completed.")

def import_xml_to_mediawiki(xml_file):
    command = f"docker-compose exec -T mediawiki php maintenance/importDump.php < {xml_file}"
    output, error, return_code = run_command(command)
    if return_code != 0:
        print(f"Error importing XML: {error}")
        sys.exit(1)
    print(output)

def main():
    if not os.path.exists('LocalSettings.php'):
        setup_mediawiki()

    dumps = get_available_dumps()
    print("Available dumps:")
    for i, dump in enumerate(dumps):
        print(f"{i + 1}. {dump}")
    
    choice = int(input("Enter the number of the dump you want to download: ")) - 1
    selected_dump = dumps[choice]
    
    files = get_dump_files(selected_dump)
    xml_file = next(iter(files))
    
    print(f"Downloading {xml_file}...")
    download_file(BASE_URL + selected_dump + '/' + xml_file, xml_file)
    
    print("Importing XML dump into MediaWiki...")
    import_xml_to_mediawiki(xml_file)
    print("Import completed successfully.")

if __name__ == "__main__":
    main()

Running the Script

python3 setup_and_import.py

Terminal Output: when script running at this stage:

noob@noob-elite:~/wiki$ python3 setup_and_import.py 
Setting up MediaWiki...
WARN[0000] /home/noob/wiki/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Running 33/2
 ✔ database Pulled                                                                                                                                                                      54.0s 
 ✔ mediawiki Pulled                                                                                                                                                                     95.0s 
[+] Running 3/3
 ✔ Network wiki_default        Created                                                                                                                                                   0.1s 
 ✔ Container wiki-database-1   Started                                                                                                                                                   0.3s 
 ✔ Container wiki-mediawiki-1  Started                                                                                                                                                   0.5s 
Please complete the MediaWiki installation by visiting http://localhost:8080
Use the following database settings:
Database host: database
Database name: my_wiki
Database username: wikiuser
Database password: wikipass
Press Enter when you have completed the installation and downloaded LocalSettings.php...

Script Execution Steps

• Initial Setup: The script starts Docker containers and waits for MediaWiki installation.
• Manual Steps:
  • Leave the script terminal running.
  • Open a new terminal for further commands.
  • Visit http://localhost:8080 in your browser to complete the MediaWiki setup using:
    • Database host: database
    • Database name: my_wiki
    • Database username: wikiuser
    • Database password: wikipass
• Post Installation:
  • Download LocalSettings.php from the browser.
  • This Downloads to your Downloads directory, move/copy to your noobwiki directory.

• • Stop the containers:
    • NOTE: docker-compose commands are ment to be run in direcotry containing the docker-compose.yml file Or you need to use docker-compose -f /path/to/docker-compose.yml down -v

docker-compose down -v

• • Uncomment the LocalSettings.php volume mapping in docker-compose.yml.

$EDITOR docker-compose.yml

docker-compose.yml After uncommenting:

services:
  database:
    image: mysql:5.7
    environment:
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: wikipass
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
    volumes:
      - ./mysql_data:/var/lib/mysql

  mediawiki:
    image: mediawiki
    ports:
      - 8080:80
    links:
      - database
    volumes:
      - ./mediawiki_data:/var/www/html/images
      - ./LocalSettings.php:/var/www/html/LocalSettings.php # This line is commented out initially

• • Restart Docker:

docker-compose up -d

• Return to Script Terminal:
  • Press Enter to continue with XML dump selection and import.

Post LocalSettings.php script

After returning to script and pressing Enter, the script will scan xml.completenoobs.com and give you a list of wiki dumps with a Number on the left and a date.

• Enter number to select a dated xml

Press Enter when you have completed the installation and downloaded LocalSettings.php...
[+] Restarting 1/1
 ✔ Container wiki-mediawiki-1  Started                                                   1.3s 
MediaWiki setup completed.
Available dumps:
1. 31_05_23.Noobs
2. 30_03_23.Noobs
3. 28_04_24.Noobs
4. 28_04_23.Noobs
5. 26_06_23.Noobs
6. 26_05_23.Noobs
7. 26_04_23.Noobs
8. 23_04_23.Noobs
9. 23_03_23.Noobs
10. 21_06_23.Noobs
11. 21_05_23.Noobs
12. 21_04_23.Noobs
13. 20_03_23.Noobs
14. 17_04_23.Noobs
15. 16_06_23.Noobs
16. 16_05_23.Noobs
17. 11_07_23.Noobs
18. 11_06_23.Noobs
19. 11_05_23.Noobs
20. 06_07_23.Noobs
21. 06_06_23.Noobs
22. 06_05_23.Noobs
23. 04_05_23.Noobs
24. 01_07_23.Noobs
25. 01_06_23.Noobs
26. 01_05_23.Noobs
Enter the number of the dump you want to download: 3
Downloading 28_04_24.Noobs.xml...
Importing XML dump into MediaWiki...
Done!
You might want to run rebuildrecentchanges.php to regenerate RecentChanges,
and initSiteStats.php to update page and revision counts

Import completed successfully.

Post-Setup Maintenance

Now, we need to rebuild our wiki so that the database reflects our imported content.

Login to Container for Maintenance

• Syntax:docker exec -it <container_name_and_service>-1 /bin/bash
  • use:docker container ls or docker ps to see container and service names.

docker exec -it noobwiki-mediawiki-1 /bin/bash

• Run maintenance scripts:

php maintenance/initSiteStats.php
php maintenance/rebuildrecentchanges.php
php maintenance/rebuildall.php

• Exit the container:

exit

Verify Wiki Content

• Check all pages have loaded by visiting:

http://127.0.0.1:8080/index.php/Special:AllPages

Here's the section in raw MediaWiki syntax:

Network Configuration

By default, your wiki might only be accessible from the host machine where Docker is running, using localhost or 127.0.0.1. However, if you want others on your network to access your wiki, you need to make some adjustments.

Current Setup: The computer running Docker Compose has an IP address of 192.168.0.44.

• You can find the IP address of your computer running Docker using the command:

ip addr

OutPut from ip addr

noob@noob-HP-EliteDesk-800-G1-DM:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: eno1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 8c:dc:d4:3d:93:49 brd ff:ff:ff:ff:ff:ff
    altname enp0s25
3: wlxe8de27142be2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e8:de:27:14:2b:e2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.44/24 brd 192.168.0.255 scope global dynamic noprefixroute wlxe8de27142be2
       valid_lft 86357sec preferred_lft 86357sec
    inet6 fe80::afbe:cc73:73a2:fcdf/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

My IP is 192.168.0.44

If someone from another computer on the network tries to visit 192.168.0.44:8080, they might encounter a "cannot connect" error. This happens because MediaWiki, by default, redirects to 127.0.0.1:8080, which is only accessible from the host machine itself.

Solution: To allow access from other devices on the same network, you need to:

• Edit the LocalSettings.php File: This isn't done inside the Docker container but rather in the directory where your docker-compose.yml file is located. Here, you need to change the server URL configuration.

• Modify URL Configuration: On lines 34-35 of LocalSettings.php, you'll find the following:

Allowing Access from Other Network Devices

• Edit LocalSettings.php in the Docker Compose directory to change the server URL:

## The protocol and server name to use in fully-qualified URLs
$wgServer = 'http://192.168.0.44:8080';

• Restart Docker to apply changes:

docker-compose restart

This allows access to your wiki from other devices on the network using 192.168.0.44:8080. This adjustment tells MediaWiki to use the network IP of the host (192.168.0.44) instead of the local loopback (127.0.0.1), allowing other devices on the network to access the wiki through 192.168.0.44:8080. Even after this change, localhost:8080 or 127.0.0.1:8080 will still work on the host machine, but now the wiki is also accessible via the network IP from other devices.