Jim's Blog https://jcarr.codeberg.page/blog/ Musings... quarto-1.9.37 Sat, 13 Sep 2025 04:00:00 GMT Memory Debugging in C Jim Carr https://jcarr.codeberg.page/blog/posts/2025/memory_debug_c.html The landscape of memory safety in low-level programming languages, particularly C and C++, is increasingly under scrutiny due to the significant security vulnerabilities associated with unsafe memory practices. Recent reports and initiatives highlight the urgent need for a shift towards memory-safe languages to mitigate these risks.

Key Findings on Memory Safety Vulnerabilities

  1. Prevalence of Memory Safety Issues:
    • A staggering 70% of vulnerabilities in products from major tech companies like Microsoft and Google are attributed to memory safety issues. This includes common problems such as buffer overflows and use-after-free bugs.
    • In mobile operating systems, studies indicate that 60-90% of vulnerabilities are related to memory safety.
  2. Types of Memory Safety Errors:
    • Out-of-Bounds Errors: Occur when a program accesses memory outside the allocated range, potentially leading to data corruption or unauthorized access.
    • Use-After-Free Bugs: Happen when a program continues to use memory after it has been freed, which can lead to unpredictable behavior and security breaches.
  3. Historical Context:
    • Notable security incidents, such as the Heartbleed vulnerability, which compromised sensitive data across numerous websites, were rooted in memory safety flaws. Such incidents underscore the critical need for improved memory management practices.

Recommendations and Initiatives

  1. Adoption of Memory-Safe Languages:
    • The U.S. White House Office of the National Cyber Director has recommended transitioning to memory-safe languages like Rust, Go, C#, Java, and Swift. These languages incorporate built-in protections that help prevent common memory-related vulnerabilities.
    • The National Security Agency (NSA) and Cybersecurity and Infrastructure Security Agency (CISA) have jointly emphasized the importance of adopting memory-safe languages as part of a comprehensive cybersecurity strategy.
  2. The Memory Safety Continuum:
    • Released by the Open Source Security Foundation (OpenSSF), this framework provides a structured approach for organizations to assess and improve their memory safety practices. It encourages incremental improvements rather than viewing memory safety as a binary state.
  3. Challenges in Transition:
    • While the shift to memory-safe languages is encouraged, organizations face challenges such as performance considerations, the complexity of legacy codebases, and the need for developer training. Immediate adoption may not be feasible, necessitating a phased approach.

But, with a massive existing codebase, along with a talent pool with many decades of experience, what steps can we take to improve the quality and safety of code in older languages, C in particular?

Let’s talk about that!

Tools

  • GCC, the GNU C compiler. It’s preinstalled on most Linux distributions.
  • Clang Tidy, provides linting and code modernization.
  • Address Sanitizer, a compile flag built into the gcc compiler.
  • Valgrind, a tool used for memory debugging, memory leak detection, and profiling.

Install Tools

sudo apt install clang-tidy valgrind

Verify that Clang Tidy and Valgrind are installed and available:

clang-tidy --version

valgrind --version

Output (your versions may differ):

Ubuntu LLVM version 18.1.3
  Optimized build.

valgrind-3.22.0

Memory Debugging

Let’s walk through some scenarios.

Buffer Overflow

buffer_overflow.c
#include <stdio.h>
#include <string.h>

void vulnerableFunction() {
    char buffer[10]; // A buffer that can hold 10 characters
    printf("Enter some text: ");
    gets(buffer); // Unsafe function that does not check buffer size
    printf("You entered: %s\n", buffer);
}

int main() {
    vulnerableFunction();
    return 0;
}

Compile it:

gcc -g -o buffer_overflow buffer_overflow.c

We immediately get a warning from the compiler, as gets() is inherently unsafe and has been deprecated in the C11 standard:

buffer_overflow.c: In function ‘vulnerableFunction’:
buffer_overflow.c:8:5: warning: implicit declaration of function ‘gets’; did you mean ‘fgets’? [-Wimplicit-function-declaration]
    8 |     gets(buffer); // Unsafe function that does not check buffer size
      |     ^~~~
      |     fgets
/usr/bin/ld: /tmp/ccbCyqjb.o: in function `vulnerableFunction':
buffer_overflow.c:(.text+0x3c): warning: the `gets' function is dangerous and should not be used.

Let’s “fix” it, but make a mistake on purpose:

buffer_overflow.c
#include <stdio.h>
#include <string.h>

void vulnerableFunction()
{
    char buffer[10];

    printf("Enter some text: ");
    fgets(buffer, 20, stdin);

    printf("You entered: %s\n", buffer);
}

int main()
{
    vulnerableFunction();

    return 0;
}

We’ve switched to the safe fgets function, but “accidently” specified a size that’s too large. When we compile:

gcc -g -o buffer_overflow_updated buffer_overflow_updated.c
buffer_overflow_updated.c: In function ‘vulnerableFunction’:
buffer_overflow_updated.c:9:5: warning: ‘fgets’ writing 20 bytes into a region of size 10 overflows the destination [-Wstringop-overflow=]
    9 |     fgets(buffer, 20, stdin);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~
buffer_overflow_updated.c:6:10: note: destination object ‘buffer’ of size 10
    6 |     char buffer[10];
      |          ^~~~~~
In file included from buffer_overflow_updated.c:1:
/usr/include/stdio.h:654:14: note: in a call to function ‘fgets’ declared with attribute ‘access (write_only, 1, 2)’
  654 | extern char *fgets (char *__restrict __s, int __n, FILE *__restrict __stream)
      |              ^~~~~

In this scenario, the compiler does a good job of catching the mistake. Let’s update it and do it the right way:

buffer_overflow.c
#include <stdio.h>
#include <string.h>

void vulnerableFunction()
{
    char buffer[10];

    printf("Enter some text: ");
    fgets(buffer, sizeof(buffer), stdin);

    printf("You entered: %s\n", buffer);
}

int main()
{
    vulnerableFunction();

    return 0;
}

The code now compiles with no errors. For this scenario, we didn’t need to use any additional tools to find the problem. But, the compiler can’t catch everything. Consider this example:

buffer_overflow_dynamic.c
#include <stdio.h>
#include <string.h>

void vulnerableFunction(int size)
{
    char buffer[10];  // A buffer that can hold 10 characters
    char input[size]; // Variable-length input buffer

    printf("Enter some text: ");

    // Unsafe use of fgets: size is determined at runtime:
    fgets(input, size, stdin); // Read input into the variable-length buffer

    // Potential buffer overflow if input exceeds buffer size
    // No bounds checking on the copy
    strcpy(buffer, input); // Unsafe copy that can overflow buffer

    printf("You entered: %s\n", buffer);
}

int main()
{
    int size = 20;

    // Call the vulnerable function with the size value:
    vulnerableFunction(size);
    return 0;
}

In the code above, the size of the input buffer is set to 20 (too large), but the compiler evaluates vulnerableFunction, where the input buffer size is treated as dynamic, and so the error isn’t caught: This code compiles with no errors or warnings, but running it gives us something like this:

Enter some text: Some text that is too large on purpose!
You entered: Some text that is t
*** stack smashing detected ***: terminated
Aborted (core dumped)

If we run the code through clang-tidy, we get some hints about unsafe behavior:

clang-tidy buffer_overflow_dynamic.c
1 warning generated.
buffer_overflow_dynamic.c:16:5: warning: Call to function 'strcpy' is insecure as it does not provide bounding of the memory buffer. Replace unbounded copy functions with analogous functions that support length arguments such as 'strlcpy'. CWE-119 [clang-analyzer-security.insecureAPI.strcpy]
   16 |     strcpy(buffer, input); // Unsafe copy that can overflow buffer
      |     ^~~~~~

If we re-compile with the address sanitizer flag:

gcc -g -fsanitize=address -g buffer_overflow_dynamic.c -o buffer_overflow_dynamic

…and then run, the failure does get reported, again with information about the point of failure:

Enter some text: Some text that is too large on purpose!
=================================================================
==1999384==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x728e9730002a at pc 0x728e998a7923 bp 0x7fff2dc230e0 sp 0x7fff2dc22888
WRITE of size 20 at 0x728e9730002a thread T0
    #0 0x728e998a7922 in strcpy ../../../../src/libsanitizer/asan/asan_interceptors.cpp:563
    #1 0x64151ad9347d in vulnerableFunction buffer_overflow_dynamic.c:16
    #2 0x64151ad9353a in main buffer_overflow_dynamic.c:26
    #3 0x728e9942a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #4 0x728e9942a28a in __libc_start_main_impl ../csu/libc-start.c:360
    #5 0x64151ad931e4 in _start (buffer_overflow_dynamic+0x11e4) (BuildId: 953df7f0607850ac9a18932ea140e86278e57fa6)

Address 0x728e9730002a is located in stack of thread T0 at offset 42 in frame
    #0 0x64151ad932b8 in vulnerableFunction buffer_overflow_dynamic.c:5

  This frame has 1 object(s):
    [32, 42) 'buffer' (line 6) <== Memory access at offset 42 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow ../../../../src/libsanitizer/asan/asan_interceptors.cpp:563 in strcpy
Shadow bytes around the buggy address:
  0x728e972ffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e972ffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e972ffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e972fff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e972fff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x728e97300000: f1 f1 f1 f1 00[02]f3 f3 00 00 00 00 00 00 00 00
  0x728e97300080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e97300100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e97300180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e97300200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x728e97300280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1999384==ABORTING

So, in this case (similar to clang-tidy) we do at least get the failure point, which gives us something to work backwards from.

Finally, if we run through Valgrind:

valgrind --leak-check=full --show-leak-kinds=all ./buffer_overflow_dynamic
==2008869== Memcheck, a memory error detector
==2008869== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==2008869== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==2008869== Command: ./buffer_overflow_dynamic
==2008869== 
Enter some text: Some text that is too long on purpose!
You entered: Some text that is t
*** stack smashing detected ***: terminated
==2008869== 
==2008869== Process terminating with default action of signal 6 (SIGABRT)
==2008869==    at 0x490EB2C: __pthread_kill_implementation (pthread_kill.c:44)
==2008869==    by 0x490EB2C: __pthread_kill_internal (pthread_kill.c:78)
==2008869==    by 0x490EB2C: pthread_kill@@GLIBC_2.34 (pthread_kill.c:89)
==2008869==    by 0x48B527D: raise (raise.c:26)
==2008869==    by 0x48988FE: abort (abort.c:79)
==2008869==    by 0x48997B5: __libc_message_impl.cold (libc_fatal.c:134)
==2008869==    by 0x49A6C18: __fortify_fail (fortify_fail.c:24)
==2008869==    by 0x49A7EA3: __stack_chk_fail (stack_chk_fail.c:24)
==2008869==    by 0x1092C8: vulnerableFunction (buffer_overflow_dynamic.c:19)
==2008869==    by 0x1092EB: main (buffer_overflow_dynamic.c:26)
==2008869== 
==2008869== HEAP SUMMARY:
==2008869==     in use at exit: 2,048 bytes in 2 blocks
==2008869==   total heap usage: 2 allocs, 0 frees, 2,048 bytes allocated
==2008869== 
==2008869== 1,024 bytes in 1 blocks are still reachable in loss record 1 of 2
==2008869==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2008869==    by 0x48F51B4: _IO_file_doallocate (filedoalloc.c:101)
==2008869==    by 0x4905523: _IO_doallocbuf (genops.c:347)
==2008869==    by 0x4902F8F: _IO_file_overflow@@GLIBC_2.2.5 (fileops.c:745)
==2008869==    by 0x4903AAE: _IO_new_file_xsputn (fileops.c:1244)
==2008869==    by 0x4903AAE: _IO_file_xsputn@@GLIBC_2.2.5 (fileops.c:1197)
==2008869==    by 0x48D0CC8: __printf_buffer_flush_to_file (printf_buffer_to_file.c:59)
==2008869==    by 0x48D0CC8: __printf_buffer_to_file_done (printf_buffer_to_file.c:120)
==2008869==    by 0x48DB742: __vfprintf_internal (vfprintf-internal.c:1545)
==2008869==    by 0x48D01B2: printf (printf.c:33)
==2008869==    by 0x10926A: vulnerableFunction (buffer_overflow_dynamic.c:9)
==2008869==    by 0x1092EB: main (buffer_overflow_dynamic.c:26)
==2008869== 
==2008869== 1,024 bytes in 1 blocks are still reachable in loss record 2 of 2
==2008869==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2008869==    by 0x48F51B4: _IO_file_doallocate (filedoalloc.c:101)
==2008869==    by 0x4905523: _IO_doallocbuf (genops.c:347)
==2008869==    by 0x4902883: _IO_file_underflow@@GLIBC_2.2.5 (fileops.c:486)
==2008869==    by 0x49055D1: _IO_default_uflow (genops.c:362)
==2008869==    by 0x48F6F79: _IO_getline_info (iogetline.c:60)
==2008869==    by 0x48F5BD3: fgets (iofgets.c:53)
==2008869==    by 0x109282: vulnerableFunction (buffer_overflow_dynamic.c:12)
==2008869==    by 0x1092EB: main (buffer_overflow_dynamic.c:26)
==2008869== 
==2008869== LEAK SUMMARY:
==2008869==    definitely lost: 0 bytes in 0 blocks
==2008869==    indirectly lost: 0 bytes in 0 blocks
==2008869==      possibly lost: 0 bytes in 0 blocks
==2008869==    still reachable: 2,048 bytes in 2 blocks
==2008869==         suppressed: 0 bytes in 0 blocks
==2008869== 
==2008869== For lists of detected and suppressed errors, rerun with: -s
==2008869== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Aborted (core dumped)

Again, we get a decent amount of detail around where things went wrong.

Overall, a tricky problem!

Out of Bounds Access

We’ll be investigating an Out of Bounds error, but there are actually two problems with this code. Do you see them?

out_of_bounds.c
#include <stdlib.h>

int main()
{
    char* p = malloc(16);
    p[42] = 100;

    return 0;
}

First, let’s see if the linter can find anything:

clang-tidy out_of_bounds.c

Output:

1 warning generated.
out_of_bounds.c:8:5: warning: Potential leak of memory pointed to by 'p' [clang-analyzer-unix.Malloc]
    8 |     return 0;
      |     ^
out_of_bounds.c:5:15: note: Memory is allocated
    5 |     char *p = malloc(16);
      |               ^~~~~~~~~~
out_of_bounds.c:8:5: note: Potential leak of memory pointed to by 'p'
    8 |     return 0;
      |     ^

Not a lot of detail, but we do get some hints about possible problems with our memory allocation.

Let’s go ahead and compile with gcc:

gcc -g -o out_of_bounds out_of_bounds.c

The code compiles with no errors. When we run, it also runs with no errors (probably!) and reports nothing. I say probably because we are making two mistakes in this code:

  1. Writing data outside the bounds of the space allocated for p, and
  2. Not freeing the allocated space after we’re done.

Let’s try again, this time with the sanitize flag:

gcc -fsanitize=address -g -o out_of_bounds out_of_bounds.c

Now when we run, we actually get some useful feedback:

==2642974==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x50200000003a at pc 0x55a1ade4e1ff bp 0x7ffcaaeb3380 sp 0x7ffcaaeb3370
WRITE of size 1 at 0x50200000003a thread T0
    #0 0x55a1ade4e1fe in main out_of_bounds.c:6
    #1 0x70fe6c22a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #2 0x70fe6c22a28a in __libc_start_main_impl ../csu/libc-start.c:360
    #3 0x55a1ade4e0e4 in _start (out_of_bounds+0x10e4) (BuildId: ec7069e9292b8acada6ab57a772da4c32c98eeeb)

0x50200000003a is located 26 bytes after 16-byte region [0x502000000010,0x502000000020)
allocated by thread T0 here:
    #0 0x70fe6c6fd9c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x55a1ade4e1be in main out_of_bounds.c:5
    #2 0x70fe6c22a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #3 0x70fe6c22a28a in __libc_start_main_impl ../csu/libc-start.c:360
    #4 0x55a1ade4e0e4 in _start (out_of_bounds+0x10e4) (BuildId: ec7069e9292b8acada6ab57a772da4c32c98eeeb)

SUMMARY: AddressSanitizer: heap-buffer-overflow out_of_bounds.c:6 in main
Shadow bytes around the buggy address:
  0x501ffffffd80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x501ffffffe00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x501ffffffe80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x501fffffff00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x501fffffff80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x502000000000: fa fa 00 00 fa fa fa[fa]fa fa fa fa fa fa fa fa
  0x502000000080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000000100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000000180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000000200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x502000000280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==2642974==ABORTING

Running with the sanitize flag, we get information about the out-of-bounds write. Highlights:

  • heap-buffer-overflow on address 0x50200000003a
  • #0 0x55a1ade4e1fe in main out_of_bounds.c:6
  • 0x50200000003a is located 26 bytes after 16-byte region [0x502000000010,0x502000000020)

With this, we actually have some actionable information. But, compiling with the sanitize flag does not identify the missing free.

Let’s see what Valgrind can find. Recompile, and then pass it to Valgrind:

gcc -g -o out_of_bounds out_of_bounds.c

valgrind --leak-check=full ./out_of_bounds

Output:

==2648121== Memcheck, a memory error detector
==2648121== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==2648121== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==2648121== Command: ./out_of_bounds
==2648121== 
==2648121== Invalid write of size 1
==2648121==    at 0x10916B: main (out_of_bounds.c:6)
==2648121==  Address 0x4a8506a is 26 bytes after a block of size 16 in arena "client"
==2648121== 
==2648121== 
==2648121== HEAP SUMMARY:
==2648121==     in use at exit: 16 bytes in 1 blocks
==2648121==   total heap usage: 1 allocs, 0 frees, 16 bytes allocated
==2648121== 
==2648121== 16 bytes in 1 blocks are definitely lost in loss record 1 of 1
==2648121==    at 0x4846828: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==2648121==    by 0x10915E: main (out_of_bounds.c:5)
==2648121== 
==2648121== LEAK SUMMARY:
==2648121==    definitely lost: 16 bytes in 1 blocks
==2648121==    indirectly lost: 0 bytes in 0 blocks
==2648121==      possibly lost: 0 bytes in 0 blocks
==2648121==    still reachable: 0 bytes in 0 blocks
==2648121==         suppressed: 0 bytes in 0 blocks
==2648121== 
==2648121== For lists of detected and suppressed errors, rerun with: -s
==2648121== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

Looking at the highlights, we see that both problems have been identified:

Error Explanation
Address 0x4a8506a is 26 bytes after a block of size 16 in arena “client” This refers to the p[42] = 100 statement. A value is being written outside of the allocated block of memory.
16 bytes in 1 blocks are definitely lost in loss record 1 of 1 We allocated a block of 16 bytes with malloc(16), but we never freed it.

Fix the code:

out_of_bounds.c
#include <stdlib.h>

int main()
{
    char *p = malloc(16);
    p[8] = 100; /* write within the allocated block */
    free(p);    /* free the allocated block*/

    return 0;
}

Compile and check it again:

gcc -o out_of_bounds out_of_bounds.c

valgrind --leak-check=full ./out_of_bounds

Output:

==2651821== Memcheck, a memory error detector
==2651821== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==2651821== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==2651821== Command: ./out_of_bounds
==2651821== 
==2651821== 
==2651821== HEAP SUMMARY:
==2651821==     in use at exit: 0 bytes in 0 blocks
==2651821==   total heap usage: 1 allocs, 1 frees, 16 bytes allocated
==2651821== 
==2651821== All heap blocks were freed -- no leaks are possible
==2651821== 
==2651821== For lists of detected and suppressed errors, rerun with: -s
==2651821== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

And now, we’re good!

What’s Next?

Some additional memory debugging topics you might want to explore:

Topic Description
Use after free This occurs when a program continues to use a memory location after it has been freed or deallocated. This can lead to undefined behavior, crashes, or security vulnerabilities, as the memory may be reallocated for other purposes or may contain invalid data.
Stack/Heap exhaustion This occurs when the call stack (a special region of memory that stores information about the active subroutines of a computer program) or heap (used for dynamic memory allocation) runs out of available space.
Null pointer dereferences This occurs when a program attempts to access or manipulate data through a pointer that has not been initialized or has been set to null.
Dangling pointers This describes a pointer that does not point to a valid object of the appropriate type. This situation arises when the object that the pointer was pointing to has been deleted or deallocated, but the pointer itself has not been updated to reflect this change. As a result, the pointer still holds the memory address of the deallocated object, leading to undefined behavior if accessed.

Also, here are some additional tools to consider:

  • Splint, a tool for statically checking C programs for security vulnerabilities and coding mistakes.
  • Cppcheck is a static analysis tool that can detect various issues in C/C++ code, including potential memory leaks.
  • Electric Fence, a linkable library that provides a malloc debugger.
  • Dr. Memory, a memory monitoring tool capable of identifying memory-related programming errors such as accesses of uninitialized memory, accesses to unaddressable memory (including outside of allocated heap units and heap underflow and overflow), accesses to freed memory, double frees, memory leaks, and (on Windows) handle leaks, GDI API usage errors, and accesses to un-reserved thread local storage slots.

Conclusion

Non-memory safe languages provide a lot of footguns, but with careful planning and judicious use of supporting tools, you can greatly improve the safety of your code.

]]> C https://jcarr.codeberg.page/blog/posts/2025/memory_debug_c.html Sat, 13 Sep 2025 04:00:00 GMT CGI Programming Jim Carr https://jcarr.codeberg.page/blog/posts/2025/cgi_programming.html In computing, Common Gateway Interface (CGI) is an interface specification that enables web servers to execute an external program to process HTTP or HTTPS user requests.

The official CGI logo from the spec announcement

The name CGI comes from the early days of the Web, where webmasters wanted to connect legacy information systems such as databases to their Web servers. The CGI program was executed by the server and provided a common “gateway” between the Web server and the legacy information system.

While it was once a dominant method for creating dynamic web content, modern web development has introduced various alternatives that often outperform CGI in terms of efficiency and scalability.

A Brief History of CGI Programming

In the early days of the web (early 1990s) CGI was introduced as a standard protocol for web servers to interact with executable programs. This allowed web servers to generate dynamic content by executing scripts or programs in response to user requests. The first widely used web server, NCSA HTTPd, implemented CGI, which quickly became a cornerstone of web development.

Language Support

Initially, CGI scripts were primarily written in C, as it was one of the most popular programming languages at the time, but many languages are supported, including:

Programming Language Description
Perl One of the most popular languages for CGI due to its text processing capabilities and ease of use.
Python Known for its readability and simplicity, Python is widely used for CGI scripting.
PHP While often used as a module within web servers, PHP can also be run as a CGI script.
Ruby Ruby can be used for CGI, especially with frameworks like Sinatra.
C/C++ These languages can be used for high-performance CGI applications, though they require more complex setup.
Shell Scripting Shell scripts can be executed as CGI scripts, particularly in Unix/Linux environments.
Java Java can be used for CGI through servlets, though it’s more common to use Java with web frameworks.
Tcl Tcl is another scripting language that can be used for CGI.

The ability to create dynamic web pages revolutionized how information was shared online, leading to the development of interactive websites and web applications.

Writing CGI

Let’s explore the process of creating and utilizing some CGI programs!

We’ll be using Python for our local development server (it supports CGI), but in keeping with the legacy spirit, we’ll use C and Perl to create our CGI.

Tip

This assumes a Linux environment, with C (gcc), Python 3, and Perl installed.

Development Server

Create a directory to hold your web page. I’ll use www_root:

mkdir www_root
Note

The standard/default directory for CGI programs is cgi-bin.

Inside www_root, create a cgi-bin directory:

cd www_root

mkdir cgi-bin

Start Python’s development server, with CGI support:

python3 -m http.server --cgi

Simple Page Render

For our first example, we’ll render an entire page using a CGI program written in C.

First create the CGI source file:

simple_cgi.c
#include <stdio.h>

int main(void) {
    // Print the HTTP header
    printf("Content-Type: text/html\n\n");

    // Print the HTML content
    printf("<html>\n");
    printf("<head><title>Simple CGI Example</title></head>\n");
    printf("<body>\n");
    printf("<h1>Hello, World!</h1>\n");
    printf("<p>This is a simple CGI program written in C.</p>\n");
    printf("</body>\n");
    printf("</html>\n");

    return 0;
}

Compile it and make sure it’s executable, then copy the binary to your cgi-bin directory:

gcc -o simple_cgi simple_cgi.c

chmod 755 simple_cgi

cp simple_cgi /path/to/project/www_root/cgi-bin

Navigate to http://127.0.0.1:8000/cgi-bin/simple_cgi in your browser, and you’ll see this:

To implement the same example in Perl, first create the Perl script in the cgi-bin directory:

simple_cgi_p.pl
#!/usr/bin/perl

use strict;
use warnings;
use CGI qw(:standard); # Include the CGI module, along with a standard set of commonly-used functions

# Create a new CGI object
my $cgi = CGI->new;

# Print the HTTP header
print $cgi->header('text/html');

# Print the HTML content
print $cgi->start_html('Simple CGI Example');
print $cgi->h1('Hello, World!');
print $cgi->p('This is a simple CGI program written in Perl.');
print $cgi->end_html;

Then, navigate to http://127.0.0.1:8000/cgi-bin/simple_cgi_p.pl in your browser, and you’ll see this:

A More Complex Example: Handling Form Posting

A prevalent application of CGI is to capture the data from web forms, which can be used for purposes such as email submissions and storing information in a database.

First, create the source file for the CGI:

form_capture.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAX_DATA_SIZE 1024

int main(void) {
    // Print the HTTP header
    printf("Content-Type: text/html\n\n");

    // Buffer to hold the input data
    char *data = getenv("CONTENT_LENGTH");
    int content_length = data ? atoi(data) : 0;

    // Check if content length is valid
    if (content_length > 0 && content_length < MAX_DATA_SIZE) {
        char input_data[MAX_DATA_SIZE];
        fread(input_data, 1, content_length, stdin);
        input_data[content_length] = '\0'; // Null-terminate the string

        // Print the HTML response
        printf("<html>\n");
        printf("<head><title>Form Data Received</title></head>\n");
        printf("<body>\n");
        printf("<h1>Form Data Received</h1>\n");
        printf("<p>Your input:</p>\n");
        printf("<pre>%s</pre>\n", input_data);
        printf("</body>\n");
        printf("</html>\n");
    } else {
        // Handle error for invalid content length
        printf("<html>\n");
        printf("<head><title>Error</title></head>\n");
        printf("<body>\n");
        printf("<h1>Error</h1>\n");
        printf("<p>Invalid input data.</p>\n");
        printf("</body>\n");
        printf("</html>\n");
    }

    return 0;
}

Compile it and make it executable, then copy the binary to your cgi-bin directory:

gcc -o form_capture form_capture.c

chmod 755 form_capture

cp form_capture /path/to/project/www_root/cgi-bin

In www_root, create an HTML file for the form:

form.html
<!DOCTYPE html>
<html>
<head>
    <title>Simple Form</title>
</head>
<body>
    <h1>Submit Your Data</h1>
    <form action="/cgi-bin/form_capture" method="post">
        <label for="data">Enter some data:</label><br>
        <input type="text" id="data" name="data"><br><br>
        <input type="submit" value="Submit">
    </form>
</body>
</html>

Navigate to http://127.0.0.1:8000/form.html and enter some data:

Click “Submit”, and you’ll see this:

Accomplishing the same thing in Perl is a bit more straightforward. In cgi-bin, create your form capture CGI script (Perl version):

form_capture_p.pl
#!/usr/bin/perl

use strict;
use warnings;
use CGI qw(:standard);

# Create a new CGI object
my $cgi = CGI->new;

# Print the HTTP header
print $cgi->header('text/html');

# Get the form data
my $data = $cgi->param('data');

# Print the HTML response
print $cgi->start_html('Form Data Received');
print $cgi->h1('Form Data Received');
print $cgi->p('Your input:');
print $cgi->pre($data // 'No data received');  # Display data or a message if no data
print $cgi->end_html;

Make the script executable:

chmod 755 form_capture_p.pl

Then, update the action attribute in form.html to use the Perl script instead:

<form action="/cgi-bin/form_capture_p.pl" method="post">

Return to your browser, refresh http://127.0.0.1:8000/form.html, enter some data, and click “Submit”. You’ll see similar results.

The Waning Popularity of CGI

Despite its historical significance, CGI programming has seen a decline in popularity for several reasons:

  • Performance Issues: CGI creates a new process for each request, which can lead to significant overhead and slower response times, especially under heavy load.
  • Emergence of New Technologies: Frameworks and technologies such as FastCGI, mod_perl, and ASP.NET have emerged, offering better performance and more features. These alternatives allow for persistent processes, reducing the overhead associated with CGI.
  • Rise of Modern Web Frameworks: Languages like JavaScript (with Node.js), Ruby on Rails, and Django have gained traction, providing developers with more powerful tools for building dynamic web applications. These frameworks often come with built-in features for routing, templating, and database interaction, making them more efficient than traditional CGI.
  • Shift to Client-Side Rendering: The rise of single-page applications (SPAs) and client-side frameworks like React and Angular has shifted the focus from server-side scripting to client-side rendering, further diminishing the role of CGI.

Conclusion

CGI programming laid the groundwork for dynamic web content and played a crucial role in the early days of the internet. While it has largely been replaced by more efficient and modern technologies, understanding its history and functionality provides valuable context for the evolution of web development. As we continue to innovate and improve web technologies, the lessons learned from CGI will remain relevant in shaping the future of web applications.

]]> Retro C Perl https://jcarr.codeberg.page/blog/posts/2025/cgi_programming.html Fri, 05 Sep 2025 04:00:00 GMT The Dark Forest Jim Carr https://jcarr.codeberg.page/blog/posts/2025/the_dark_forest.html The Dark Forest Theory is a fascinating concept that delves into the reasons behind the apparent silence of the universe. It suggests that the cosmos is akin to a dark forest, where every civilization is a silent hunter, hiding from others to avoid potential threats.

A dark forest

The Fermi Paradox

The Fermi Paradox refers to the apparent contradiction between the high probability of extraterrestrial life in the universe and the lack of evidence or contact with such civilizations. Named after physicist Enrico Fermi, the paradox raises the question: “If the universe is so vast and old, where is everybody?”

Key Points of the Fermi Paradox

  • Vastness of the Universe: The universe contains billions of galaxies, each with billions of stars, many of which have planets in the habitable zone where life could potentially exist.
  • Age of the Universe: The universe is approximately 13.8 billion years old, providing ample time for intelligent life to develop and potentially become advanced enough to communicate or travel across space.
  • Drake Equation: This equation estimates the number of active, communicative extraterrestrial civilizations in the Milky Way galaxy. While the equation suggests a high number of potential civilizations, we have yet to find any.
  • Lack of Evidence: Despite extensive searches, such as the SETI (Search for Extraterrestrial Intelligence) program, we have not detected any signals or signs of intelligent life.

Possible Explanations for the Paradox

The Dark Forest Theory offers a possible explanation for the Fermi Paradox. It is one of several:

  • Rare Earth Hypothesis: Suggests that the conditions for life are extremely rare, making Earth an exceptional case.
  • Great Filter: Proposes that there is a stage in the evolution of life that is extremely hard to surpass, which could be behind us (e.g., the emergence of intelligent life) or ahead of us (e.g., self-destruction).
  • Technological Limitations: Advanced civilizations may use technologies that are beyond our current understanding, making their signals undetectable.
  • Self-Destruction: Civilizations may tend to self-destruct before they can establish contact with others, due to wars, environmental collapse, or other catastrophic events.
  • Non-Interference: Advanced civilizations might choose not to contact us, adhering to a principle of non-interference with less developed societies.
  • Transient Civilizations: Civilizations may be short-lived, arising and falling before they can communicate with others.

Understanding the Dark Forest Theory

The Dark Forest Theory was popularized by the Chinese science fiction author Liu Cixin in his novel “The Dark Forest,” which is the second book in the “Three-Body Problem” trilogy. The theory posits that:

  • Civilizations are like hunters: In a dark forest, every civilization is a hunter trying to survive. They must remain silent to avoid detection by others who may pose a threat.
  • The universe is dangerous: The theory suggests that any civilization that reveals its location risks being destroyed by more advanced civilizations. This creates a paradox where civilizations choose to remain quiet rather than risk annihilation.
  • Communication is risky: Sending signals into space could attract unwanted attention, leading to a preemptive strike from a more advanced civilization.

Implications of the Dark Forest Theory

  • Fermi Paradox: As I mentioned above, the Dark Forest Theory offers a potential solution to the Fermi Paradox, which questions why we have not yet encountered extraterrestrial life despite the vastness of the universe. If civilizations are deliberately silent, it explains the lack of contact.
  • Humanity’s Future: As humanity continues to explore space, the theory raises ethical questions about our own communication efforts. Should we broadcast our presence, or is silence the safer option?
  • Existential Risks: The theory highlights the existential risks associated with advanced technology. As civilizations develop, the potential for conflict increases, leading to a cycle of fear and silence.

Critiques and Counterarguments

While the Dark Forest Theory is compelling, it is not without its critiques:

  • Optimism about Cooperation: Some argue that civilizations may be more inclined to cooperate rather than compete. The potential for mutual benefit could encourage communication rather than silence.
  • Technological Limitations: The theory assumes that all civilizations are technologically advanced enough to pose a threat. However, many civilizations may be at different stages of development, leading to a more complex interaction landscape.
  • The Nature of Life: The theory is based on human assumptions about survival and competition. Other forms of life may have different motivations and behaviors that do not align with the hunter-prey dynamic.

Conclusion

The Dark Forest Theory invites us to ponder the nature of existence and the potential for life beyond Earth. It challenges our understanding of communication, survival, and the universe itself. As we continue to explore the cosmos, the questions raised by this theory will remain relevant, urging us to consider the implications of our actions in the vast, dark forest of the universe.

]]> Astronomy https://jcarr.codeberg.page/blog/posts/2025/the_dark_forest.html Tue, 02 Sep 2025 04:00:00 GMT Evaluating Zig Jim Carr https://jcarr.codeberg.page/blog/posts/2025/evaluating_zig.html The results of the 2025 Stack Overflow Developer Survey were recently published and I noticed that interest in the Zig programming language is rising. I’d heard of Zig, but knew nothing about the details.

Let’s check it out!

What is Zig?

A summary from the Zig home page and other sources:

Zig is a general-purpose programming language that emphasizes performance, safety, and maintainability. It was created by Andrew Kelley and has gained traction for its ability to provide low-level control over system resources while maintaining a modern syntax and features that enhance developer productivity. Zig is particularly well-suited for systems programming, game development, and applications where performance is critical.

Key Features of Zig

  • Manual Memory Management: Zig allows developers to manage memory manually, similar to C, but with added safety features. This gives developers fine-grained control over memory allocation and deallocation, which is essential for performance-critical applications.
  • Error Handling: Zig introduces a unique approach to error handling that avoids exceptions and instead uses a simple return value mechanism. This makes it easier to reason about error states and ensures that errors are handled explicitly.
  • Cross-Compilation: Zig is designed with cross-compilation in mind, making it easy to build applications for different platforms from a single codebase. This is particularly useful for developers targeting multiple operating systems or architectures.
  • Interoperability with C: Zig can directly call C functions and use C libraries without the need for wrappers or bindings. This makes it easy to integrate Zig into existing C projects or leverage existing C libraries.

Why Choose Zig?

Zig stands out in a crowded field of programming languages for several reasons:

  • Performance: With its low-level capabilities and manual memory management, Zig is designed for high-performance applications, making it an excellent choice for systems programming and game development.
  • Simplicity: Zig’s syntax is clean and straightforward, making it accessible for new developers while still providing the power needed for experienced programmers.
  • Safety: The language’s focus on safety features, such as compile-time checks and explicit error handling, helps prevent common programming errors that can lead to crashes or security vulnerabilities.
  • Active Community: Zig has a growing community of developers who contribute to its development and provide support through forums, documentation, and tutorials.

Installation (Direct Download)

  1. Download the latest version from https://ziglang.org/download/
  2. Extract and move to ~/bin/zig/
  3. Add Zig to the PATH in .profile: export PATH=$PATH:~/bin/zig

Test the compiler:

mkdir hello-world

cd hello-world

zig init

zig build run

Output:

All your codebase are belong to us.
Run `zig build test` to run the tests.

VS Code

Install the Zig Language extension. It includes a code runner, code debugger, and test runners.

Tip

Install the extension (along with any other supporting extensions) inside a separate profile dedicated solely to Zig development.

If you aren’t already using profiles in VS Code, I highly recommend it. Profiles make it easy to construct curated development environments and are also great for evaluating new extensions.

Get Started

Hello!

Let’s start with the simplest possible example:

hello.zig
const std = @import("std");

pub fn main() !void {
    const stdout = std.io.getStdOut().writer();

    try stdout.print("Hello, World!\n", .{});
}

Since this is a standalone source file (not a project), we use build-exe instead of build:

zig build-exe hello.zig

This produces two files:

  • hello.o (object file), and
  • hello (executable)

Code Breakdown

Importing the Standard Library

const std = @import("std");

This line imports the standard library of Zig, which provides various functionalities, including input/output operations, memory management, and more. The std constant will be used to access these functionalities.

Main Function Definition

pub fn main() !void {

This line defines the main function, which is the entry point of the program. The pub keyword makes it public, allowing it to be called from outside the module. The !void return type indicates that the function can return an error (denoted by !) or nothing (void).

Getting the Standard Output Writer

const stdout = std.io.getStdOut().writer();

Here, the program retrieves the standard output stream using std.io.getStdOut(). The writer() method is called on this output stream to get a writer object that can be used to print text to the console.

Printing to Standard Output

try stdout.print("Hello, World!\n", .{});

This line attempts to print the string “Hello, World!” followed by a newline character (\n) to the standard output. The try keyword is used to handle any potential errors that may occur during the printing process. If an error occurs, it will propagate up to the caller of main.

Performance and Safety

Zig has four build modes:

Parameter Debug ReleaseSafe ReleaseFast ReleaseSmall
Optimizations - improve speed, harm debugging, harm compile time On On On
Runtime Safety Checks - harm speed, harm size, crash instead of undefined behavior On On

Regardless of the build mode, here’s what an integer overflow looks like at compile time:

overflow.zig
pub fn main() !void {
    const x: u8 = 255;

    // The underscore indicates that the value is unused:
    _ = x + 1;
}
zig build-exe overflow.zig
overflow.zig:4:11: error: overflow of integer type 'u8' with value '256'
    _ = x + 1;
        ~~^~~

Zig also has a test keyword that can be used to define unit tests:

overflow.zig
test "integer overflow at compile time" {
    const x: u8 = 255;

    _ = x + 1;
}

With zig test, zig will locate all unit tests and execute them:

zig test overflow.zig 
overflow.zig:4:11: error: overflow of integer type 'u8' with value '256'
    _ = x + 1;
        ~~^~~

Manual Memory Management

Manual memory management in Zig is a key feature that allows developers to have fine-grained control over memory allocation and deallocation. Unlike languages with automatic garbage collection, Zig requires programmers to explicitly manage memory.

Here’s an example:

allocator.zig
const std = @import("std");

pub fn main() !void {
    const allocator = std.heap.page_allocator;

    var number = try allocator.alloc(i32, 1);

    number[0] = 42;

    std.debug.print("The value is: {}\n", .{number[0]});

    allocator.free(number);
}

Output:

The value is: 42

Code Breakdown

Importing Standard Library

const std = @import("std");

This line imports the standard library, allowing access to various utilities, including memory management functions.

Main Function

pub fn main() !void {

The main function is the entry point of the program. The !void return type indicates that the function can return an error.

Allocating Memory

const allocator = std.heap.page_allocator;

Here, the code defines allocator as the page allocator from the standard library, which is used for allocating memory.

var number = try allocator.alloc(i32, 1);

This line allocates memory for a single integer (i32). The try keyword is used to handle any potential errors that may occur during allocation. If the allocation fails, the function will return an error.

Setting and Printing the Value

number[0] = 42;

After successfully allocating memory, this line sets the value of the allocated integer to 42. The allocated memory is treated as an array, so number[0] accesses the first (and only) element.

std.debug.print("The value is: {}\n", .{number[0]});

This line prints the value of the integer to the console. The {} is a placeholder for the value, which is provided in the second argument as a tuple.

Deallocating Memory

allocator.free(number);

Finally, this line deallocates the memory that was previously allocated for the integer.

Important

With great power comes great responsibility!

It’s important to free allocated memory to prevent memory leaks. Explicit allocation and deallocation makes it easy to see exactly what the code’s doing, but this places the burden of responsible memory management on you!

Object Orientation

In Zig, classes as found in object-oriented languages like C++ or Java do not exist. Instead, Zig uses a combination of structs and functions to achieve similar functionality. (This is very similar to Rust.)

You can define a struct to hold data and then create functions that operate on that data, effectively mimicking class behavior:

struct.zig
const std = @import("std");

pub const MyClass = struct {
    value: i32,

    pub fn new(value: i32) MyClass {
        return MyClass{ .value = value };
    }

    pub fn increment(self: *MyClass) void {
        self.value += 1;
    }

    pub fn getValue(self: *MyClass) i32 {
        return self.value;
    }
};

pub fn main() void {
    var myObject = MyClass.new(10);
    myObject.increment();
    const currentValue = myObject.getValue();
    std.debug.print("Current Value: {}\n", .{currentValue});
}

Output:

Current Value: 11

Code Breakdown

Defining MyClass

pub const MyClass = struct {
    value: i32,

Here, MyClass is defined as a public structure (struct) that contains a single field, value, which is of type i32 (32-bit integer).

Constructor Method

pub fn new(value: i32) MyClass {
    return MyClass{ .value = value };
}
  • This is a public function new that acts as a constructor for MyClass.
  • It takes an i32 parameter value and returns a new instance of MyClass initialized with that value.

Increment Method

pub fn increment(self: *MyClass) void {
    self.value += 1;
}
  • The increment function is a public method that takes a mutable pointer to MyClass (self: *MyClass).
  • It increments the value field of the instance by 1.

Getter Method

pub fn getValue(self: *MyClass) i32 {
    return self.value;
}
  • The getValue function is a public method that returns the current value of the value field.
  • It also takes a mutable pointer to MyClass.

Main Function

pub fn main() void {
    var myObject = MyClass.new(10);
    myObject.increment();
    const currentValue = myObject.getValue();
    std.debug.print("Current Value: {}\n", .{currentValue});
}
  • The main function is the entry point of the program.
  • It creates an instance of MyClass called myObject, initialized with the value 10.
  • It calls the increment method on myObject, which increases its value from 10 to 11.
  • Finally, it retrieves the current value using getValue and prints it to the console using std.debug.print.

Cross-Compilation

Returning to our “Hello World” program:

hello.zig
const std = @import("std");

pub fn main() !void {
    const stdout = std.io.getStdOut().writer();

    try stdout.print("Hello, World!\n", .{});
}

Since I’m running in Linux, building without specifying a target produces a Linux binary:

zig build-exe hello.zig

file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped

But, we can easily target other architectures:

zig build-exe hello.zig -target x86_64-windows

file hello.exe
hello.exe: PE32+ executable (console) x86-64, for MS Windows, 7 sections

Weaknesses and Limitations

Zig is a fairly new language. I expect that many of these issues will be resolved or improved over time. But, for the time being, there are quite a few issues to consider.

Limited Ecosystem and Libraries

  • Fewer Libraries: Compared to more established languages like Python or Java, Zig has a smaller ecosystem of libraries and frameworks. This can make it challenging to find pre-built solutions for common problems.
  • Community Size: The community is still growing, which means fewer resources, tutorials, and community support compared to more mature languages.

Learning Curve

  • Syntax and Concepts: While Zig aims for simplicity, its syntax and concepts may be unfamiliar to developers coming from other languages, particularly those used to garbage-collected languages.
  • Manual Memory Management: Zig requires developers to manage memory manually, which can lead to increased complexity and potential for errors, especially for those not accustomed to this approach.

Tooling and IDE Support

  • Limited IDE Support: The tooling and IDE support for Zig are not as robust as those for more established languages. This can affect productivity, especially for developers who rely heavily on integrated development environments. (The Zig extension in VS Code does provide a decent experience, though.)
  • Build System: Zig’s build system is still evolving, and while it offers unique features, it may not be as mature or user-friendly as those found in other languages.

Summary

I’m not sure it’s ready for “prime time”, but it’s definitely an interesting language and ecosystem. I’ll be thinking about some small starter projects I can use to get more familiar with it and test its limitations.

]]> Zig https://jcarr.codeberg.page/blog/posts/2025/evaluating_zig.html Sat, 16 Aug 2025 04:00:00 GMT Learning Fortran Jim Carr https://jcarr.codeberg.page/blog/posts/2025/learning_fortran.html I recently caught up with an old friend who asked about the personal projects I’ve been involved in. I shared that I’ve been working on my Practical Astronomy language ports and recently completed C++, PHP, and Java. I expressed that I felt I had wrapped up that project since there weren’t any other languages I was eager to port. However, I jokingly suggested that “maybe I should think about Fortran.”

I kept pondering that idea and eventually thought, “Why not?”

History

Fortran is one of the oldest programming languages, with a first proposal being submitted by John Backus in 1953, and the first compiler delivered in 1957.

Fortran has had many major revisions, including:

Version Released Major features
Fortran II 1958 Procedural programming (subroutines and functions)
Fortran IV 1962 Removal of machine-dependent features; added boolean expressions; added logical IF statement
Fortran 66 1966 Standardization
Fortran 77 1978 IF / END IF blocks; DO loops; direct-access file I/O
Fortran 90 1991 Free-form source input; lowercase keywords; inline comments
Fortran 95 1997 FORALL and nested WHERE constructs; default initializations; removed some obsolete features
Fortran 2003 2004 Object-oriented programming support; C language interoperability
Fortran 2008 2010 Sub-modules; parallel execution model
Fortran 2018 2018 Improved interoperability with C; additional parallel features
Fortran 2023 2023 Addresses minor error corrections and omissions for Fortran 2018

The Port

I’ve only just begun the Fortran version of Practical Astronomy, so I don’t have a lot to share about that. But, with Fortran being a new language for me, I thought I’d share some of the challenges and idiosyncrasies I’ve encountered.

Coincidentally, I’ve kicked off this project on Easter Sunday 2025, and the first algorithm to port happens to be “date of Easter”. This algorithm takes a year value as input, and produces a full date as a result. For example, given an input year of 2025, the algorithm should give a result of 4/20/2025.

Since the result has three values, the first thing I needed to consider was how to return them: Either as a complex custom type, or as individual values (like a tuple). I decided to implement both to see how they compare in terms of complexity.

Complex Custom Type

Custom Type

The custom type used to hold our return value looks like this:

type :: CivilDate
   integer :: month
   real :: day
   integer :: year
end type

The type statement indicates that we’re creating a complex data structure that can encapsulate multiple data items of different types. The CivilDate type contains three data items: month and year are integers, and day is a real, since we sometimes need access to fractional parts of days.

The Function

Using the custom type in a function is a little tricky, since Fortran functions require custom types to be returned as pointers. Here’s the complete function:

function get_date_of_easter(input_year) result(custom_obj)
   type(CivilDate), pointer :: custom_obj
   integer :: input_year
   real :: year, a, b, c, d, e, f, g, h, i, k, l, m, n, p, day, month
 
   allocate(custom_obj)
 
   year = input_year
   a = mod(year, 19.0)
   b = floor(year / 100)
   c = mod(year, 100.0)
   d = floor(b / 4)
   e = mod(b , 4.0)
   f = floor((b + 8) / 25)
   g = floor((b - f + 1) / 3)
   h = mod( ((19 * a) + b - d - g + 15) , 30.0)
   i = floor(c / 4)
   k = mod(c , 4.0)
   l = mod( (32 + 2 * (e + i) - h - k) , 7.0)
   m = floor((a + (11 * h) + (22 * l)) / 451)
   n = floor((h + l - (7 * m) + 114) / 31)
   p = mod((h + l - (7 * m) + 114) , 31.0)
 
   day = p + 1
   month = n
 
   custom_obj%month = floor(month)
   custom_obj%day = day
   custom_obj%year = input_year
end function

Here’s what’s going on in the first four lines. First, we declare a variable named custom_obj as a pointer to an instance of the custom CivilDate type:

type(CivilDate), pointer :: custom_obj

Next, we declare the input_year parameter variable:

integer :: input_year

Then, a series of variables that will be used in calculations:

real :: year, a, b, c, d, e, f, g, h, i, k, l, m, n, p, day, month

Finally, we allocate space for custom_object, similar to how we’d use malloc in C:

allocate(custom_obj)

The next few lines use a series of (mostly) modulo and floor operations to calculate the date values:

year = input_year
a = mod(year, 19.0)
b = floor(year / 100)
c = mod(year, 100.0)
d = floor(b / 4)
e = mod(b , 4.0)
f = floor((b + 8) / 25)
g = floor((b - f + 1) / 3)
h = mod( ((19 * a) + b - d - g + 15) , 30.0)
i = floor(c / 4)
k = mod(c , 4.0)
l = mod( (32 + 2 * (e + i) - h - k) , 7.0)
m = floor((a + (11 * h) + (22 * l)) / 451)
n = floor((h + l - (7 * m) + 114) / 31)
p = mod((h + l - (7 * m) + 114) , 31.0)
 
day = p + 1
month = n

Finally, we assign the calculated date elements to the corresponding date items in our pointer object:

custom_obj%month = floor(month)
custom_obj%day = day
custom_obj%year = input_year

Calling the Function

First, we declare two variables, an integer we’ll use to provide the input year, and a pointer to hold the result:

integer :: input_year
type(CivilDate),pointer :: civil_date

Next, we call the function and assign the result:

civil_date => get_date_of_easter(input_year)

Finally, we print our results, and then free the pointer:

print *,civil_date%month
print *,civil_date%day
print *,civil_date%year
 
deallocate(civil_date)

Given an input year of 2025, here is our output:

           4
   20.0000000    
        2025

This is the complete subroutine that handles our test:

subroutine test_date_of_easter(input_year)
   integer :: input_year
   type(CivilDate),pointer :: civil_date
 
   civil_date => get_date_of_easter(input_year)
 
   print *,civil_date%month
   print *,civil_date%day
   print *,civil_date%year
 
   deallocate(civil_date)
end subroutine

Individual Values

If we prefer not to use a complex type, we can return individual values instead. Fortran doesn’t support tuples, but it does allow you to specify arguments to a subroutine as being input or output values through the use of the intent keyword.

The Subroutine

Here’s the complete subroutine:

subroutine get_date_of_easter(input_year, month, day, year)
   integer, intent(in) :: input_year
   integer, intent(out) :: month
   real, intent(out) :: day
   integer, intent(out) :: year
 
   real :: year1, a, b, c, d, e, f, g, h, i, k, l, m, n, p
 
   year1 = input_year
   a = mod(year1, 19.0)
   b = floor(year1 / 100)
   c = mod(year1, 100.0)
   d = floor(b / 4)
   e = mod(b , 4.0)
   f = floor((b + 8) / 25)
   g = floor((b - f + 1) / 3)
   h = mod(((19 * a) + b - d - g + 15) , 30.0)
   i = floor(c / 4)
   k = mod(c, 4.0)
   l = mod((32 + 2 * (e + i) - h - k) , 7.0)
   m = floor((a + (11 * h) + (22 * l)) / 451)
   n = floor((h + l - (7 * m) + 114) / 31)
   p = mod((h + l - (7 * m) + 114) , 31.0)
 
   day = p + 1
   month = floor(n)
   year = input_year
end subroutine

In the first line, we indicate that input_year is an input value:

integer, intent(in) :: input_year

Next, we indicate that month, day, and year are output values (values that will be returned to calling code):

integer, intent(out) :: month
real, intent(out) :: day
integer, intent(out) :: year

In the next series of lines, we declare variables that will be used for our internal calculations, and perform those calculations:

real :: year1, a, b, c, d, e, f, g, h, i, k, l, m, n, p
 
year1 = input_year
a = mod(year1, 19.0)
b = floor(year1 / 100)
c = mod(year1, 100.0)
d = floor(b / 4)
e = mod(b , 4.0)
f = floor((b + 8) / 25)
g = floor((b - f + 1) / 3)
h = mod(((19 * a) + b - d - g + 15) , 30.0)
i = floor(c / 4)
k = mod(c, 4.0)
l = mod((32 + 2 * (e + i) - h - k) , 7.0)
m = floor((a + (11 * h) + (22 * l)) / 451)
n = floor((h + l - (7 * m) + 114) / 31)
p = mod((h + l - (7 * m) + 114) , 31.0)

Finally, we assign the output/return values:

day = p + 1
month = floor(n)
year = input_year

Calling the Subroutine

First, we declare a variable to hold our input value:

! input
integer :: input_year

In Fortran, lines prefixed with ! are treated as comments.

Next, the variables to hold the results:

! outputs
integer :: month
real :: day
integer :: year

Then, we call the subroutine, and print the results:

call get_date_of_easter(input_year, month, day, year)
 
print *, month
print *, day
print *, year

Given an input of 2025, this is our output:

           4
   20.0000000    
        2025

This is the complete subroutine that handles our test:

subroutine test_date_of_easter(input_year)
   ! input
   integer :: input_year
 
   ! outputs
   integer :: month
   real :: day
   integer :: year
 
   call get_date_of_easter(input_year, month, day, year)
 
   print *, month
   print *, day
   print *, year
end subroutine

My Thoughts

I think that the second option (using intent) is the most concise and easiest to understand, although the first option wasn’t too bad once I grasped the syntax of the pointer.

Now I have to decide if I want to proceed with the rest of the Practical Astronomy port. This feels like it will be a pretty heavy lift.

]]> Retro Fortran https://jcarr.codeberg.page/blog/posts/2025/learning_fortran.html Sun, 20 Apr 2025 04:00:00 GMT Observing Plan for the April Astronomy Program at Garber Forest Jim Carr https://jcarr.codeberg.page/blog/posts/2025/garber_observing_plan.html

IMPORTANT ANNOUNCEMENT Sadly, the weather got us again. With rain and thunderstorms in the forecast, we’re forced to cancel the astronomy program for 4/5.

On April 5th, 2025, I will be at the Garber Forest near Lewisburg Ohio, equipped with my telescope to discuss astronomy and lead an exploration of the night sky - weather permitting, of course! (The first attempt didn’t work out.)

This event is organized by the Preble County Park District.

What We’ll Look For

These are our main targets, but we’ll look for other things too!

8pm

The sky will still be in twilight, making it difficult to spot many celestial objects, but the moon will already make a great target!

The moon will be slightly over half full and the craters along the terminator line will be in shadow, making them stand out.

A terminator line is the boundary that separates the illuminated side from the dark side of a celestial body, such as the Earth or the Moon. It marks where day transitions into night and is often associated with twilight conditions.

9pm

Rising in the East: Arcturus, part of the constellation Bootes.

Overhead, Mars will be close to the Moon.

To the West, the constellation Orion, with its bright stars Betelgeuse and Rigel and the Orion nebula.

To the left of Orion is the bright star Sirius, part of the constellation Canis Major.

To the right of Orion is the bright star Aldebaran, part of the constellation Taurus.

Just above Aldebaran is Jupiter, and to the right of Aldebaran is the Pleiades, a star cluster.

To the North, the “dippers” (Ursa Minor and Ursa Major) and Cassiopeia.

More Information

Magnitude

In astronomy, “magnitude” refers to the brightness of celestial objects, such as stars, planets, and galaxies. There are two main types of magnitude:

  1. Apparent Magnitude: This measures how bright an object appears from Earth. The scale is logarithmic and inversely related, meaning that lower numbers indicate brighter objects. For example, a star with an apparent magnitude of 1 is brighter than a star with an apparent magnitude of 6. The scale also includes negative values for very bright objects, such as the Sun and the Moon.

  2. Absolute Magnitude: This measures the intrinsic brightness of a celestial object, defined as how bright the object would appear if it were located at a standard distance of 10 parsecs (about 32.6 light-years) from Earth. Absolute magnitude allows astronomers to compare the true brightness of different objects without the effects of distance.

The concept of magnitude is crucial for understanding the luminosity and distance of celestial bodies, as well as for classifying stars and other astronomical phenomena.

Constellation

A constellation is a recognized pattern of stars in the night sky that forms a specific shape or figure, often representing mythological characters, animals, or objects. Constellations serve as a way to organize and identify groups of stars, making it easier for astronomers and stargazers to navigate the sky.

There are 88 officially recognized constellations, as defined by the International Astronomical Union (IAU). These constellations cover the entire celestial sphere and are used in various ways, including for navigation, storytelling, and as a framework for locating celestial objects. Some well-known constellations include Orion, Ursa Major, and Cassiopeia.

Asterism

In astronomy, “asterism” refers to a recognizable pattern or group of stars that may not necessarily form a formal constellation. Asterisms can be made up of stars from one or more constellations and are often more easily recognizable or familiar to the casual observer.

For example, the Big Dipper is an asterism that is part of the larger constellation Ursa Major. Similarly, the Summer Triangle is another well-known asterism formed by three bright stars from three different constellations: Vega (Lyra), Deneb (Cygnus), and Altair (Aquila).

Asterisms are useful for amateur astronomers and stargazers as they can serve as reference points for locating other stars and celestial objects in the night sky.

Altitude and Azimuth

Altitude and azimuth are two coordinates used in astronomy to specify the position of an object in the sky as observed from a specific location on Earth.

Altitude and Azimuth

  1. Altitude: This is the angle between the object in the sky and the observer’s local horizon. It is measured in degrees, ranging from 0° (the horizon) to 90° (the zenith, which is directly overhead). An object with an altitude of 0° is on the horizon, while an object with an altitude of 90° is directly above the observer.

  2. Azimuth: This is the angle measured clockwise from a reference direction, typically true north. Azimuth is also measured in degrees, ranging from 0° to 360°. For example, an azimuth of 0° corresponds to north, 90° to east, 180° to south, and 270° to west.

Together, altitude and azimuth provide a way to pinpoint the location of celestial objects in the sky relative to an observer’s position on Earth. This system is particularly useful for navigation, stargazing, and astronomical observations.

The Moon

The Moon is Earth’s only natural satellite and is the fifth largest moon in the solar system. It has a significant impact on Earth, influencing tides and stabilizing the planet’s axial tilt. The Moon’s surface is covered with craters, mountains, and flat plains called maria, which were formed by ancient volcanic activity.

The Moon has a synchronous rotation, meaning it rotates on its axis in the same time it takes to orbit Earth, which is why we always see the same side from our planet. The far side of the Moon, often mistakenly called the “dark side,” is not permanently dark; it just faces away from Earth.

The Moon has been a subject of human fascination for centuries, inspiring myths, art, and scientific exploration. The Apollo missions in the late 1960s and early 1970s marked the first time humans set foot on the Moon, with Apollo 11 being the most famous mission, landing Neil Armstrong and Buzz Aldrin on its surface in 1969.

Moon Phases

Waxing and waning refer to the phases of the moon as it orbits the Earth, specifically how the visible portion of the moon changes over time.

  1. Waxing: This term describes the period when the moon is increasing in illumination. After the new moon phase, the moon begins to wax, meaning more of its surface becomes visible from Earth. The phases during this period include:
    • Waxing Crescent: A small sliver of the moon is illuminated.
    • First Quarter: Half of the moon is illuminated.
    • Waxing Gibbous: More than half of the moon is illuminated but not yet full.
  2. Waning: This term describes the period when the moon is decreasing in illumination. After the full moon phase, the moon begins to wane, meaning less of its surface is visible. The phases during this period include:
    • Waning Gibbous: More than half of the moon is illuminated but decreasing.
    • Last Quarter: Half of the moon is illuminated, but the opposite side compared to the first quarter.
    • Waning Crescent: A small sliver of the moon is illuminated, leading back to the new moon phase.

These cycles occur roughly every 29.5 days, which is the time it takes for the moon to complete one full orbit around the Earth and return to the same phase.

Planets

There are eight recognized planets in the solar system, which are divided into two categories:

  • Terrestrial planets: Mercury, Venus, Earth, and Mars. These are rocky and have solid surfaces.
  • Gas giants: Jupiter and Saturn, which are primarily composed of hydrogen and helium.
  • Ice giants: Uranus and Neptune, which have a larger proportion of “ices” such as water, ammonia, and methane.

Dwarf Planets: These include Pluto, Eris, Haumea, Makemake, and Ceres. Dwarf planets are similar to regular planets but do not clear their orbital paths of other debris.

Jupiter

Jupiter is the largest planet in our solar system, known for its massive size, strong magnetic field, and distinctive features. Here are some key points about Jupiter:

  1. Size and Composition: Jupiter is a gas giant, primarily composed of hydrogen and helium. It has a diameter of about 86,881 miles (139,822 kilometers), making it more than 11 times wider than Earth.

  2. Great Red Spot: One of Jupiter’s most famous features is the Great Red Spot, a giant storm that has been raging for at least 350 years. It is larger than Earth and is characterized by its reddish color.

  3. Moons: Jupiter has a large number of moons, with 79 confirmed as of now. The four largest moons, known as the Galilean moons, are Io, Europa, Ganymede, and Callisto. Ganymede is the largest moon in the solar system.

  4. Rings: Jupiter has a faint ring system composed mainly of dust particles and small rocks. The rings are not as prominent as those of Saturn.

  5. Magnetic Field: Jupiter has the strongest magnetic field of any planet in the solar system, which is generated by the motion of metallic hydrogen in its interior.

  6. Exploration: Several spacecraft have visited Jupiter, including the Galileo orbiter and the Juno spacecraft, which is currently studying the planet’s atmosphere, magnetic field, and structure.

  7. Atmosphere: Jupiter’s atmosphere is known for its colorful bands and storms, including the aforementioned Great Red Spot. The planet’s atmosphere is dynamic and experiences high-speed winds.

Jupiter’s Moons

Callisto

Callisto is one of the largest moons of Jupiter and is the third-largest moon in the solar system. It has a diameter of about 4,820 kilometers (2,995 miles) and is known for its heavily cratered surface, which suggests it has been geologically inactive for a long time. Callisto is composed primarily of ice and rock, and its surface features include a mix of old impact craters and some younger, less cratered regions.

One of the most interesting aspects of Callisto is its potential subsurface ocean. While there is no direct evidence of this ocean, some scientists believe that beneath its icy crust, there may be a salty ocean that could harbor conditions suitable for life.

Callisto has a very thin atmosphere, primarily composed of carbon dioxide, and it has a low level of radiation compared to other Galilean moons, making it a more favorable location for future exploration and potential human missions. It was discovered by Galileo Galilei in 1610, along with the other three largest moons of Jupiter: Io, Europa, and Ganymede.

Europa

Europa is one of Jupiter’s largest moons and is the sixth-largest moon in the solar system. It is particularly interesting to scientists because it is believed to have a subsurface ocean beneath its icy crust, which could potentially harbor conditions suitable for life. Here are some key features of Europa:

  1. Surface and Composition: Europa’s surface is primarily composed of water ice, and it features a smooth, bright appearance with few impact craters, suggesting a relatively young surface. The cracks and ridges on its surface indicate tectonic activity.

  2. Subsurface Ocean: Scientists believe that beneath Europa’s icy shell lies a salty ocean that may be in contact with the moon’s rocky mantle. This ocean could provide the necessary chemical ingredients for life.

  3. Potential for Life: The combination of liquid water, a stable energy source (from tidal heating due to Jupiter’s gravitational pull), and essential chemical elements makes Europa a prime candidate in the search for extraterrestrial life.

  4. Exploration: Europa has been studied by several spacecraft, including the Galileo orbiter and the Hubble Space Telescope.

  5. Atmosphere: Europa has a very thin atmosphere composed mainly of oxygen, but it is far too thin to support human life.

Europa continues to be a focus of astrobiological research and exploration due to its intriguing characteristics and the possibility of finding life beyond Earth.

Europa Clipper, a space probe developed by NASA, is currently on its way to Europa and is expected to arrive in the Jupiter system in 2030.

The goals of Europa Clipper are to explore Europa, investigate its habitability, and aid in the selection of a landing site for the proposed Europa Lander. This exploration is focused on understanding the three main requirements for life: liquid water, chemistry, and energy. Specifically, the objectives are to study:

  • Ice shell and ocean: Confirm the existence and characterize the nature of water within or beneath the ice, and study processes of surface-ice-ocean exchange.
  • Composition: Distribution and chemistry of key compounds and the links to ocean composition.
  • Geology: Characteristics and formation of surface features, including sites of recent or current activity.

source

Ganymede

Ganymede is the largest moon of Jupiter and the largest moon in the solar system. It is one of the four Galilean moons, which were discovered by Galileo Galilei in 1610. Ganymede has a diameter of about 5,268 kilometers (3,273 miles), making it even larger than the planet Mercury, although it does not have enough mass to be classified as a planet.

Ganymede is unique among moons because it has a magnetic field, which is likely generated by a liquid iron or iron-sulfide core. The surface of Ganymede is a mix of two types of terrain: bright, icy regions and darker, heavily cratered areas. It is believed to have a subsurface ocean beneath its icy crust, which raises the possibility of conditions that could support life.

In addition to its geological features, Ganymede has a thin atmosphere composed mostly of oxygen, although it is far too thin to support human life. The moon has been studied by various spacecraft, including the Galileo orbiter and the Hubble Space Telescope.

Io

Io is one of the four largest moons of Jupiter. It is the most geologically active body in the solar system, with hundreds of volcanoes, some of which are still erupting. Io’s surface is covered with sulfur and sulfur dioxide, giving it a colorful appearance with shades of yellow, red, and white.

The intense geological activity on Io is primarily due to the gravitational pull from Jupiter and the other Galilean moons (Europa, Ganymede, and Callisto), which creates tidal heating. This process generates enough internal heat to drive volcanic activity.

Io has a thin atmosphere composed mainly of sulfur dioxide, and its surface temperature can vary widely. The moon also has a very weak magnetic field and is known to have a significant interaction with Jupiter’s magnetosphere.

Mars

Mars is the fourth planet from the Sun in our solar system and is often referred to as the “Red Planet” due to its reddish appearance, which is caused by iron oxide (rust) on its surface. Here are some key facts about Mars:

  1. Atmosphere: Mars has a thin atmosphere, composed mostly of carbon dioxide, with traces of nitrogen and argon. This thin atmosphere contributes to its cold temperatures and makes it difficult for liquid water to exist on the surface.

  2. Surface Features: Mars is home to the largest volcano in the solar system, Olympus Mons, and a massive canyon system called Valles Marineris. The planet also has polar ice caps that grow and recede with the changing seasons.

  3. Water: Evidence suggests that liquid water once flowed on Mars, and there are signs of ancient riverbeds and lake beds. Currently, water exists mostly in the form of ice, and there are indications of briny liquid water in some areas.

  4. Exploration: Mars has been the target of numerous robotic missions, including orbiters, landers, and rovers. Notable missions include NASA’s Mars rovers, such as Spirit, Opportunity, Curiosity, and Perseverance, which have been studying the planet’s geology and searching for signs of past life.

  5. Potential for Life: Scientists are particularly interested in Mars because of its potential to have supported microbial life in the past. Ongoing research aims to understand the planet’s habitability.

  6. Future Missions: There are plans for future missions to Mars, including potential human exploration. NASA’s Artemis program aims to return humans to the Moon as a stepping stone for future crewed missions to Mars.

Bright Stars

Aldebaran

Aldebaran is a prominent star located in the constellation Taurus. It is often referred to as the “Eye of the Bull” and is one of the brightest stars in the night sky. Aldebaran is classified as a red giant star and is approximately 65 light-years away from Earth. It has a distinct orange hue and is part of the Hyades star cluster, although it is not physically part of the cluster itself. Aldebaran is notable for its brightness and is often used in navigation and astronomy.

Arcturus

Arcturus is one of the brightest stars in the night sky and is located in the constellation Bootes. It is classified as a red giant star and is approximately 36.7 light-years away from Earth. Arcturus is notable for its orange hue, which is a result of its cooler surface temperature compared to hotter stars like our Sun. The name “Arcturus” comes from the Greek word meaning “bear watcher,” as it is associated with the nearby constellation Ursa Major (the Great Bear).

In addition to its brightness, Arcturus is also significant in astronomy for its role in the study of stellar evolution, as it represents a later stage in the life cycle of a star.

Betelgeuse

Betelgeuse is a red supergiant star located in the constellation Orion. It is one of the largest and most luminous stars known, with a diameter estimated to be about 1,000 times that of the Sun. Betelgeuse is notable for its distinct reddish color and is often used as a reference point in the night sky.

As a supergiant, Betelgeuse is in the later stages of its stellar evolution and is expected to eventually explode as a supernova. This event could happen within the next million years, although it’s difficult to predict exactly when.

In addition to its astronomical significance, Betelgeuse has captured the imagination of many cultures and is often featured in literature and popular media.

Betelgeuse was in the news recently.

Polaris

“Polaris,” also known as the North Star, is the brightest star in the constellation Ursa Minor. It is located nearly directly above the North Pole, making it a crucial point of reference for navigation in the Northern Hemisphere. Polaris is a supergiant star and is approximately 433 light-years away from Earth.

Rigel

Rigel is a prominent star located in the constellation Orion. It is one of the brightest stars in the night sky and is classified as a blue supergiant. Rigel is approximately 860 light-years away from Earth and has a surface temperature of around 12,000 Kelvin, which gives it its blue color. It is often used in navigation and is notable for its brightness and distinctive position in the Orion constellation.

Sirius

Sirius is the brightest star in the night sky and is part of the constellation Canis Major. It is often referred to as the “Dog Star” because of its position in this constellation, which represents a dog. Sirius is actually a binary star system, consisting of two stars: Sirius A, a main-sequence star, and Sirius B, a white dwarf. The star is located about 8.6 light-years from Earth and has been significant in various cultures throughout history, often associated with myths and agricultural calendars.

Deep Sky Objects

Orion Nebula

The Orion Nebula, also known as M42, is one of the most famous and studied nebulae in the night sky. It is located in the Milky Way, approximately 1,344 light-years away from Earth in the Orion constellation. The nebula is a stellar nursery, where new stars are being born from the surrounding gas and dust.

Key features of the Orion Nebula include:

  1. Brightness: It is one of the brightest nebulae visible to the naked eye and can be seen as a fuzzy patch in the sword of Orion.

  2. Size: The Orion Nebula spans about 24 light-years across and contains a vast amount of gas and dust.

  3. Star Formation: The nebula is home to a young cluster of stars known as the Trapezium, which consists of four massive stars that illuminate the surrounding gas and dust.

  4. Observation: The Orion Nebula is a popular target for amateur astronomers and astrophotographers due to its brightness and the intricate structures within it.

  5. Scientific Importance: It provides valuable insights into the processes of star formation and the dynamics of interstellar matter.

The Orion Nebula is a stunning example of the beauty and complexity of the universe, making it a favorite subject for both professional and amateur astronomers.

The Pleiades

The Pleiades, also known as the Seven Sisters, is a star cluster located in the constellation Taurus. It is one of the nearest star clusters to Earth and is easily visible to the naked eye. The cluster contains several hundred stars, but the seven brightest are typically the ones referred to as the “Seven Sisters.” These stars are named Alcyone, Asterope, Electra, Maia, Merope, Taygeta, and Pleione.

The Pleiades has been significant in various cultures throughout history, often associated with mythology and folklore. In astronomy, it is known for its beauty and is often used as a reference point for navigation. The cluster is also of interest to astronomers studying star formation and the evolution of stars.

]]> Astronomy https://jcarr.codeberg.page/blog/posts/2025/garber_observing_plan.html Sat, 01 Mar 2025 05:00:00 GMT Implement a REST Service on a Nano RP2040 Jim Carr https://jcarr.codeberg.page/blog/posts/2025/nano_rp2040_rest.html The Nano RP2040 is an Arduino Nano board with a Raspberry Pi RP2040 microcontroller that supports Bluetooth, WiFi, and machine learning. It has an onboard accelerometer, gyroscope, microphone, and temperature sensor.

With the built-in sensors and WiFi, it’s a good candidate for a remote monitoring solution. We’ll implement a simple REST service that returns the current temperature.

NOTE: I’m maintaining and improving a more feature-rich version of this project here.

Toolset

Here’s what I’ll be using:

Board Setup

Plug in your Nano RP2040 board and issue the following command:

arduino-cli board list

You should see something like this:

Port         Protocol Type              Board Name                  FQBN                                Core
/dev/ttyACM0 serial   Serial Port (USB) Arduino Nano RP2040 Connect arduino:mbed_nano:nanorp2040connect arduino:mbed_nano

You will probably need to install the core:

arduino-cli core install arduino:mbed_nano

We’ll be using three additional libraries:

Name Description
Arduino_LSM6DSOX Access the IMU for accelerometer, gyroscope, and embedded temperature sensor.
ArduinoJson A simple and efficient JSON library for embedded C++.
WiFiNINA With this library you can instantiate Servers, Clients and send/receive UDP packets through WiFi.

Install them:

arduino-cli lib install Arduino_LSM6DSOX

arduino-cli lib install ArduinoJson

arduino-cli lib install WiFiNINA

Project Structure / Setup

Create a directory named web_server_rest_temp.

Open the directory in VS Code.

Create a Makefile containing the following content:

ARDCMD=arduino-cli
FQBNSTR=arduino:mbed_nano:nanorp2040connect
PORT=/dev/ttyACM0
SKETCHNAME=web_server_rest_temp

default:
    @echo 'Targets:'
    @echo '  compile  -- Compile sketch, but don''t upload it.'
    @echo '  upload   -- Compile and upload sketch.'
    @echo '  monitor  -- Open the serial port monitor.'

compile:
    $(ARDCMD) compile --fqbn $(FQBNSTR) $(SKETCHNAME)

upload: compile
    $(ARDCMD) upload -p $(PORT) --fqbn $(FQBNSTR) $(SKETCHNAME)

monitor:
    $(ARDCMD) monitor -p $(PORT)

You can type out individual commands for compiling, uploading, and monitoring instead, but using a Makefile is more convenient.

Create a subdirectory, also named web_server_rest_temp.

In the subdirectory, create a file named arduino_secrets.h containing the following contents:

#define SECRET_SSID "wireless_network_name"           // The name of your wireless network
#define SECRET_PASSWORD "wireless_network_password"   // The password for your wireless network
#define SECRET_PORT 8090                              // The port that the REST service will run on

Update the defines to match your network settings.

For example, if you have a wireless network named “MyHomeWifi” and you use the password “MyN3tw0rkP@ssw0rd” to connect to it, your arduino_secrets.h file would look like this:

#define SECRET_SSID "MyHomeWifi"              // The name of your wireless network
#define SECRET_PASSWORD "MyN3tw0rkP@ssw0rd"   // The password for your wireless network
#define SECRET_PORT 8090                      // The port that the REST service will run on

In the same subdirectory, create a file named web_server_rest_temp.ino and give it the boilerplate Arduino contents:

void setup()
{

}

void loop()
{

}

Your project layout should now look like this:

Keep web_server_rest_temp.ino open for editing.

Main Sketch

We’ll update web_server_rest_temp.ino incrementally and explain each update, then show a complete version at the end.

First, add your includes:

#include "arduino_secrets.h"
#include <Arduino_LSM6DSOX.h>
#include <ArduinoJson.h>
#include <WiFiNINA.h>

Then, initialize network settings:

char ssid[] = SECRET_SSID;     // network SSID (name)
char pass[] = SECRET_PASSWORD; //  network password
int keyIndex = 0;              // network key index number (needed only for WEP)
int port = SECRET_PORT;

Set the initial status and instantiate the server:

int status = WL_IDLE_STATUS;
WiFiServer server(port);

Inside setup(), initialize serial communication at 9600 baud. (The serial monitor uses this)

Serial.begin(9600);

Make sure the IMU is available:

if (!IMU.begin())
{
    Serial.println("Failed to initialize IMU!");
    while (1)
        ;
}

An inertial measurement unit (IMU) is an electronic device that measures and reports a body’s specific force, angular rate, and sometimes the orientation of the body, using a combination of accelerometers, gyroscopes, and sometimes magnetometers. When the magnetometer is included, IMUs are referred to as IMMUs.

Make sure the WiFi module is available:

if (WiFi.status() == WL_NO_MODULE)
{
    Serial.println("Communication with WiFi module failed!");
    while (true)
        ;
}

See if the WiFi firmware is up to date:

String fv = WiFi.firmwareVersion();

if (fv < WIFI_FIRMWARE_LATEST_VERSION)
{
    Serial.println("Please upgrade the firmware");
}

If it isn’t, then display a message, but still continue.

Connect to the WiFi network:

while (status != WL_CONNECTED)
{
    Serial.print("Attempting to connect to Network named: ");
    Serial.println(ssid); // print the network name (SSID)

    status = WiFi.begin(ssid, pass);

    // wait 5 seconds for the connection:
    delay(5000);
}

Start the server, then print the WiFi status:

server.begin();

printWifiStatus();

The printWifiStatus() function is a new custom function that we need to add. It displays useful information about our new connection:

void printWifiStatus()
{
    // print the SSID of the network you're attached to:
    Serial.print("SSID: ");
    Serial.println(WiFi.SSID());

    // print your board's IP address:
    IPAddress ip = WiFi.localIP();
    Serial.print("IP Address: ");
    Serial.println(ip);

    // print the received signal strength:
    long rssi = WiFi.RSSI();
    Serial.print("signal strength (RSSI): ");
    Serial.print(rssi);
    Serial.println(" dBm");
}

That concludes our setup() code. Now we’re ready to move on to loop().

First, instantiate a WiFiClient. It will listen for incoming clients:

WiFiClient client = server.available();

Check to see when a client connects:

if (client)
{

}

When a client connects, print a message, then initialize a String that will hold incoming data from the client:

if (client)
{
    Serial.println("new client");

    String currentLine = "";
}

Prepare to perform some operations while the client is connected, but only while the client is still available:

if (client)
{
    Serial.println("new client");

    String currentLine = "";

    while (client.connected())
    {
        if (client.available())
        {
        }
    }
}

Inside the client.available() block:

// Read one character of the client request at a time:
char c = client.read();

// If the byte is a newline character:
if (c == '\n')
{
    // If currentline has been cleared, the request is finished, but it wasn't a known endpoint, so send a generic response:
    if (currentLine.length() == 0)
    {
        sendResponse(client, "Hello from Arduino RP2040! Valid endpoints are /Temperature/Current/F and /Temperature/Current/C", -99, "invalid");
        break;
    }
    else
        currentLine = ""; // If you got a newline, then clear currentLine
}
else if (c != '\r')
    // If you got anything else but a carriage return character, add it to the end of the currentLine:
    currentLine += c;

sendResponse() is another new custom function. It receives the following as arguments:

  1. A reference to the WiFiClient,
  2. a text message,
  3. a value, and
  4. a status

Inside the function:

  1. A standard HTTP response is sent.
  2. A JSON document object is created.
  3. The JSON document is populated with the message, value, and status.
  4. The JSON object is serialized back to the client.
void sendResponse(WiFiClient &client, char message[], int value, char status[])
{
    // Send a standard HTTP response
    client.println("HTTP/1.1 200 OK");
    client.println("Content-type: application/json");
    client.println("Connection: close");
    client.println();

    // Create a JSON object
    StaticJsonDocument<200> doc;

    doc["message"] = message;
    doc["value"] = value;
    doc["status"] = status;

    // Serialize JSON to client
    serializeJson(doc, client);
}

Returning to the loop() function, still inside the client.available() block, we now check to see if a specific endpoint was called by the client:

char request_unit = 'X';
if (currentLine.indexOf("GET /Temperature/Current/F") != -1)
    request_unit = 'F';
if (currentLine.indexOf("GET /Temperature/Current/C") != -1)
    request_unit = 'C';

We use request_unit to track calls to specific endpoints. If the client has asked for the current temperature in Fahrenheit or Celsius, we’ll want to respond accordingly:

if (request_unit == 'F' || request_unit == 'C')
{
    int current_temperature = (request_unit == 'F') ? getTemperature(true) : getTemperature(false);
    char temp_units[5];
    sprintf(temp_units, "°%s", (request_unit == 'F') ? "F" : "C");

    char message[50];
    sprintf(message, "Current temperature is %d %s", current_temperature, temp_units);

    sendResponse(client, message, current_temperature, "success");
    break;
}

If the client has asked for the temperature, we retrieve it with getTemperature(), format the data to return to the client, then send the response.

getTemperature() is another new function. It makes sure the IMU module is available, then retrieves the current temperature value from the IMU temperature sensor. The IMU returns the temperature in Celsius units, so the value is converted to Fahrenheit, if requested.

int getTemperature(bool as_fahrenheit)
{
    if (IMU.temperatureAvailable())
    {
        int temperature_deg = 0;
        IMU.readTemperature(temperature_deg);

        if (as_fahrenheit == true)
            temperature_deg = celsiusToFahrenheit(temperature_deg);

        return temperature_deg;
    }
    else
    {
        return -99;
    }
}

The celsiusToFahrenheit() function is also new:

int celsiusToFahrenheit(int celsius_value)
{
  return (celsius_value * (9 / 5)) + 32;
}

Returning to the loop() function, after the while (client.connected()) block, we perform some cleanup after the client disconnects:

client.stop();
Serial.println("client disconnected");

And that’s it! Our full sketch now looks like this:

#include "arduino_secrets.h"
#include <Arduino_LSM6DSOX.h>
#include <ArduinoJson.h>
#include <WiFiNINA.h>

char ssid[] = SECRET_SSID;     // network SSID (name)
char pass[] = SECRET_PASSWORD; // network password
int keyIndex = 0;              // network key index number (needed only for WEP)
int port = SECRET_PORT;

void setup()
{
    Serial.begin(9600);

    if (!IMU.begin())
    {
        Serial.println("Failed to initialize IMU!");
        while (1)
            ;
    }

    if (WiFi.status() == WL_NO_MODULE)
    {
        Serial.println("Communication with WiFi module failed!");
        while (true)
            ;
    }

    String fv = WiFi.firmwareVersion();

    if (fv < WIFI_FIRMWARE_LATEST_VERSION)
    {
        Serial.println("Please upgrade the firmware");
    }

    while (status != WL_CONNECTED)
    {
        Serial.print("Attempting to connect to Network named: ");
        Serial.println(ssid); // print the network name (SSID);

        // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
        status = WiFi.begin(ssid, pass);
        // wait 5 seconds for connection:
        delay(5000);
    }

    server.begin();
    printWifiStatus();
}

void loop()
{
    WiFiClient client = server.available();

    if (client)
    {
        Serial.println("new client");

        String currentLine = "";

        while (client.connected())
        {
            if (client.available())
            {
                // Read one character of the client request at a time:
                char c = client.read();

                // If the byte is a newline character:
                if (c == '\n')
                {
                    // If currentline has been cleared, the request is finished, but it wasn't a known endpoint, so send a generic response:
                    if (currentLine.length() == 0)
                    {
                        sendResponse(client, "Hello from Arduino RP2040! Valid endpoints are /Temperature/Current/F and /Temperature/Current/C", -99, "invalid");
                        break;
                    }
                    else
                        currentLine = ""; // If you got a newline, then clear currentLine
                }
                else if (c != '\r')
                    // If you got anything else but a carriage return character, add it to the end of the currentLine:
                    currentLine += c;

                char request_unit = 'X';
                if (currentLine.indexOf("GET /Temperature/Current/F") != -1)
                    request_unit = 'F';
                if (currentLine.indexOf("GET /Temperature/Current/C") != -1)
                    request_unit = 'C';

                if (request_unit == 'F' || request_unit == 'C')
                {
                    int current_temperature = (request_unit == 'F') ? getTemperature(true) : getTemperature(false);
                    char temp_units[5];
                    sprintf(temp_units, "°%s", (request_unit == 'F') ? "F" : "C");

                    char message[50];
                    sprintf(message, "Current temperature is %d %s", current_temperature, temp_units);

                    sendResponse(client, message, current_temperature, "success");
                    break;
                }
            }
        }

        client.stop();
        Serial.println("client disconnected");
    }
}

void sendResponse(WiFiClient &client, char message[], int value, char status[])
{
    // Send a standard HTTP response
    client.println("HTTP/1.1 200 OK");
    client.println("Content-type: application/json");
    client.println("Connection: close");
    client.println();

    // Create a JSON object
    StaticJsonDocument<200> doc;

    doc["message"] = message;
    doc["value"] = value;
    doc["status"] = status;

    // Serialize JSON to client
    serializeJson(doc, client);
}

void printWifiStatus()
{
    // print the SSID of the network you're attached to:
    Serial.print("SSID: ");
    Serial.println(WiFi.SSID());

    // print your board's IP address:
    IPAddress ip = WiFi.localIP();
    Serial.print("IP Address: ");
    Serial.println(ip);

    // print the received signal strength:
    long rssi = WiFi.RSSI();
    Serial.print("signal strength (RSSI): ");
    Serial.print(rssi);
    Serial.println(" dBm");
}

int getTemperature(bool as_fahrenheit)
{
    if (IMU.temperatureAvailable())
    {
        int temperature_deg = 0;
        IMU.readTemperature(temperature_deg);

        if (as_fahrenheit == true)
            temperature_deg = celsiusToFahrenheit(temperature_deg);

        return temperature_deg;
    }
    else
    {
        return -99;
    }
}

Compile and Upload

Make sure the board is connected, then open a terminal.

Compile:

make compile

or

arduino-cli compile --fqbn arduino:mbed_nano:nanorp2040connect web_server_rest_temp

You should see something similar to this:

Sketch uses 112214 bytes (0%) of program storage space. Maximum is 16777216 bytes.
Global variables use 44552 bytes (16%) of dynamic memory, leaving 225784 bytes for local variables. Maximum is 270336 bytes.

Used library     Version
Arduino_LSM6DSOX 1.1.2
Wire
SPI
ArduinoJson      7.3.0
WiFiNINA         1.9.0

Used platform     Version
arduino:mbed_nano 4.2.1

Upload:

make upload

or

arduino-cli upload -p /dev/ttyACM0 --fqbn arduino:mbed_nano:nanorp2040connect web_server_rest_temp

You should see something similar to this:

...
New upload port: /dev/ttyACM0 (serial)

Start the monitor to check the status of the running server:

make monitor

or

arduino-cli monitor -p /dev/ttyACM0

Results will be similar to this:

Using default monitor configuration for board: arduino:mbed_nano:nanorp2040connect
Monitor port settings:
  baudrate=9600
  bits=8
  dtr=on
  parity=none
  rts=on
  stop_bits=1

Connecting to /dev/ttyACM0. Press CTRL-C to exit.
SSID: (network name)
IP Address: (server ip address)
signal strength (RSSI): -40 dBm

Call the Service

Now that we’ve completed our code, flashed the device, and our server is running, we’re ready to test it.

First, note the server address from the monitor above. I’ll use an example of 192.168.0.186.

There are many options for calling the service. You could use cURL:

curl --request GET --url http://192.168.0.186:8090/Temperature/Current/F

If you’re using a REST runner that recognizes .http files, your request will look something like this:

GET http://192.168.0.186:8090/Temperature/Current/F

You could also use a REST client like Postman or Insomnia. Since these are simple GET requests, you can even put the URL directly into a web browser. Regardless of how you call the service, though, you should see a response similar to this:

HTTP/1.1 200 OK
Content-type: application/json
Connection: close

{
  "message": "Current temperature is 70 °F",
  "value": 70,
  "status": "success"
}

If you call the service with an endpoint it doesn’t recognize, you’ll see this:

HTTP/1.1 200 OK
Content-type: application/json
Connection: close

{
  "message": "Hello from Arduino RP2040! Valid endpoints are /Temperature/Current/F and /Temperature/Current/C",
  "value": -99,
  "status": "invalid"
}

Next Steps

Now that your temperature monitoring service is up and running, a fun next step might be to go fully remote. You can easily test this by using a charger block. For example, I plugged my Nano RP2040 into a BLAVOR Solar Charger Power Bank.

]]> C Embedded and IoT https://jcarr.codeberg.page/blog/posts/2025/nano_rp2040_rest.html Sun, 02 Feb 2025 05:00:00 GMT ESP8266 MicroPython Project with OLED Jim Carr https://jcarr.codeberg.page/blog/posts/2025/esp8266_oled.html In my last post, we walked through setup of an ESP8266 project managed with UV. Now we’ll construct a more complex project, adding a simple OLED display.

Hardware

Parts needed:

  • esp8266 board
  • OLED 0.96 inch Display
  • jumper wires
  • breadboard

(All of this is included in the ESP32 Basic Starter Kit).

Follow this schematic:

Credit: https://randomnerdtutorials.com/micropython-oled-display-esp32-esp8266/

Connections:

OLED ESP8266
VCC 3.3V
GND GND
SCL GPIO 5 (D1)
SDA GPIO 4 (D2)

Software

Initialize Project

mkdir esp8266_oled

cd esp8266_oled

uv init

Add Dependencies

uv add esptool

uv add adafruit-ampy

Make sure you can communicate with the esp8266:

uv tool run --from esptool esptool.py chip_id

Flash MicroPython to the ESP8266

wget https://micropython.org/resources/firmware/ESP8266_GENERIC-20241129-v1.24.1.bin

uv tool run --from esptool esptool.py erase_flash

uv tool run --from esptool esptool.py --baud 460800 write_flash --flash_size=detect 0 ESP8266_GENERIC-20241129-v1.24.1.bin

Create Scripts

The MicroPython standard library doesn’t include support for the OLED by default. We’ll create a new ssd1306.py file and update it with the required code.

touch ssd1306.py

Copy the contents of https://github.com/RuiSantosdotme/ESP-MicroPython/raw/master/code/Others/OLED/ssd1306.py into the file:

# MicroPython SSD1306 OLED driver, I2C and SPI interfaces created by Adafruit

import time
import framebuf

# register definitions
SET_CONTRAST        = const(0x81)
SET_ENTIRE_ON       = const(0xa4)
SET_NORM_INV        = const(0xa6)
SET_DISP            = const(0xae)
SET_MEM_ADDR        = const(0x20)
SET_COL_ADDR        = const(0x21)
SET_PAGE_ADDR       = const(0x22)
SET_DISP_START_LINE = const(0x40)
SET_SEG_REMAP       = const(0xa0)
SET_MUX_RATIO       = const(0xa8)
SET_COM_OUT_DIR     = const(0xc0)
SET_DISP_OFFSET     = const(0xd3)
SET_COM_PIN_CFG     = const(0xda)
SET_DISP_CLK_DIV    = const(0xd5)
SET_PRECHARGE       = const(0xd9)
SET_VCOM_DESEL      = const(0xdb)
SET_CHARGE_PUMP     = const(0x8d)


class SSD1306:
    def __init__(self, width, height, external_vcc):
        self.width = width
        self.height = height
        self.external_vcc = external_vcc
        self.pages = self.height // 8
        # Note the subclass must initialize self.framebuf to a framebuffer.
        # This is necessary because the underlying data buffer is different
        # between I2C and SPI implementations (I2C needs an extra byte).
        self.poweron()
        self.init_display()

    def init_display(self):
        for cmd in (
            SET_DISP | 0x00, # off
            # address setting
            SET_MEM_ADDR, 0x00, # horizontal
            # resolution and layout
            SET_DISP_START_LINE | 0x00,
            SET_SEG_REMAP | 0x01, # column addr 127 mapped to SEG0
            SET_MUX_RATIO, self.height - 1,
            SET_COM_OUT_DIR | 0x08, # scan from COM[N] to COM0
            SET_DISP_OFFSET, 0x00,
            SET_COM_PIN_CFG, 0x02 if self.height == 32 else 0x12,
            # timing and driving scheme
            SET_DISP_CLK_DIV, 0x80,
            SET_PRECHARGE, 0x22 if self.external_vcc else 0xf1,
            SET_VCOM_DESEL, 0x30, # 0.83*Vcc
            # display
            SET_CONTRAST, 0xff, # maximum
            SET_ENTIRE_ON, # output follows RAM contents
            SET_NORM_INV, # not inverted
            # charge pump
            SET_CHARGE_PUMP, 0x10 if self.external_vcc else 0x14,
            SET_DISP | 0x01): # on
            self.write_cmd(cmd)
        self.fill(0)
        self.show()

    def poweroff(self):
        self.write_cmd(SET_DISP | 0x00)

    def contrast(self, contrast):
        self.write_cmd(SET_CONTRAST)
        self.write_cmd(contrast)

    def invert(self, invert):
        self.write_cmd(SET_NORM_INV | (invert & 1))

    def show(self):
        x0 = 0
        x1 = self.width - 1
        if self.width == 64:
            # displays with width of 64 pixels are shifted by 32
            x0 += 32
            x1 += 32
        self.write_cmd(SET_COL_ADDR)
        self.write_cmd(x0)
        self.write_cmd(x1)
        self.write_cmd(SET_PAGE_ADDR)
        self.write_cmd(0)
        self.write_cmd(self.pages - 1)
        self.write_framebuf()

    def fill(self, col):
        self.framebuf.fill(col)

    def pixel(self, x, y, col):
        self.framebuf.pixel(x, y, col)

    def scroll(self, dx, dy):
        self.framebuf.scroll(dx, dy)

    def text(self, string, x, y, col=1):
        self.framebuf.text(string, x, y, col)


class SSD1306_I2C(SSD1306):
    def __init__(self, width, height, i2c, addr=0x3c, external_vcc=False):
        self.i2c = i2c
        self.addr = addr
        self.temp = bytearray(2)
        # Add an extra byte to the data buffer to hold an I2C data/command byte
        # to use hardware-compatible I2C transactions.  A memoryview of the
        # buffer is used to mask this byte from the framebuffer operations
        # (without a major memory hit as memoryview doesn't copy to a separate
        # buffer).
        self.buffer = bytearray(((height // 8) * width) + 1)
        self.buffer[0] = 0x40  # Set first byte of data buffer to Co=0, D/C=1
        self.framebuf = framebuf.FrameBuffer1(memoryview(self.buffer)[1:], width, height)
        super().__init__(width, height, external_vcc)

    def write_cmd(self, cmd):
        self.temp[0] = 0x80 # Co=1, D/C#=0
        self.temp[1] = cmd
        self.i2c.writeto(self.addr, self.temp)

    def write_framebuf(self):
        # Blast out the frame buffer using a single I2C transaction to support
        # hardware I2C interfaces.
        self.i2c.writeto(self.addr, self.buffer)

    def poweron(self):
        pass


class SSD1306_SPI(SSD1306):
    def __init__(self, width, height, spi, dc, res, cs, external_vcc=False):
        self.rate = 10 * 1024 * 1024
        dc.init(dc.OUT, value=0)
        res.init(res.OUT, value=0)
        cs.init(cs.OUT, value=1)
        self.spi = spi
        self.dc = dc
        self.res = res
        self.cs = cs
        self.buffer = bytearray((height // 8) * width)
        self.framebuf = framebuf.FrameBuffer1(self.buffer, width, height)
        super().__init__(width, height, external_vcc)

    def write_cmd(self, cmd):
        self.spi.init(baudrate=self.rate, polarity=0, phase=0)
        self.cs.high()
        self.dc.low()
        self.cs.low()
        self.spi.write(bytearray([cmd]))
        self.cs.high()

    def write_framebuf(self):
        self.spi.init(baudrate=self.rate, polarity=0, phase=0)
        self.cs.high()
        self.dc.high()
        self.cs.low()
        self.spi.write(self.buffer)
        self.cs.high()

    def poweron(self):
        self.res.high()
        time.sleep_ms(1)
        self.res.low()
        time.sleep_ms(10)
        self.res.high()

Create and update main.py. This is our entry point.

touch main.py
  1. Copy contents of https://github.com/RuiSantosdotme/ESP-MicroPython/raw/master/code/Others/OLED/main.py into the file, then
  2. Comment the esp32 pin assignment and uncomment the esp8266 pin assignment

Contents should end up like this:

# Complete project details at https://RandomNerdTutorials.com/micropython-programming-with-esp32-and-esp8266/

from machine import Pin, SoftI2C
import ssd1306
from time import sleep

# ESP32 Pin assignment
# i2c = SoftI2C(scl=Pin(22), sda=Pin(21)) # esp32

# ESP8266 Pin assignment
i2c = SoftI2C(scl=Pin(5), sda=Pin(4))

oled_width = 128
oled_height = 64
oled = ssd1306.SSD1306_I2C(oled_width, oled_height, i2c)

oled.text('Hello, World!', 0, 0)
oled.text('Hello, World 2!', 0, 10)
oled.text('Hello, World 3!', 0, 20)

oled.show()

Upload Scripts to ESP3266

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 put ssd1306.py

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 put main.py

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 ls

Run the Main Script

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 run main.py

On the OLED display you should see this:

Hello, World!
Hello, World 2!
Hello, World 3!

You can learn more about what the code is doing here: https://randomnerdtutorials.com/micropython-oled-display-esp32-esp8266/

]]> Python Embedded and IoT https://jcarr.codeberg.page/blog/posts/2025/esp8266_oled.html Sun, 12 Jan 2025 05:00:00 GMT Using UV To Manage a ESP8266 MicroPython Project Jim Carr https://jcarr.codeberg.page/blog/posts/2025/esp8266_uv.html Following up from my previous UV post, today I’ll walk through managing an ESP8266 microcontroller project with UV. I’ll stick to the basics with this one, but I’ll implement a more complex project in a future article.

Background

The ESP8266 is a low-cost Wi-Fi microcontroller, with built-in TCP/IP networking software, and microcontroller capability, produced by Espressif Systems in Shanghai, China.

The chip was popularized in the English-speaking maker community in August 2014 via the ESP-01 module, made by a third-party manufacturer Ai-Thinker. This small module allows microcontrollers to connect to a Wi-Fi network and make simple TCP/IP connections using Hayes-style commands. However, at first, there was almost no English-language documentation on the chip and the commands it accepted. The very low price and the fact that there were very few external components on the module, which suggested that it could eventually be very inexpensive in volume, attracted many hackers to explore the module, the chip, and the software on it, as well as to translate the Chinese documentation.

from Wikipedia

I’m using the ESP32 Basic Starter Kit, which contains an ESP8266 module with the following specifications:

Description Value
Model ESP8266MOD
Vendor AI-THINKER
ISM 2.4 GHz
PA +25 dBm
Wireless 802.11b/g/n

Project Setup

Adapted from here: https://docs.micropython.org/en/latest/esp8266/tutorial/intro.html

Make sure your ESP8266 is connected.

Initialize Project, Install ESP Tools

Create a project directory and cd into it:

mkdir esp_test
 
cd esp_test

Initialize the project:

uv init

The esptool utility is used to communicate with the ROM bootloader in Expressif chips.

Add the esptool package:

uv add esptool

Show available commands:

uv tool run --from esptool esptool.py

Commands:

Command Name Description
load_ram Download an image to RAM and execute
dump_mem Dump arbitrary memory to disk
read_mem Read arbitrary memory location
write_mem Read-modify-write to arbitrary memory location
write_flash Write a binary blob to flash
run Run application code in flash
image_info Dump headers from a binary file (bootloader or application)
make_image Create an application image from binary files
elf2image Create an application image from ELF file
read_mac Read MAC address from OTP ROM
chip_id Read Chip ID from OTP ROM
flash_id Read SPI flash manufacturer and device ID
read_flash_status Read SPI flash status register
write_flash_status Write SPI flash status register
read_flash Read SPI flash content
verify_flash Verify a binary blob against flash
erase_flash Perform Chip Erase on SPI flash
erase_region Erase a region of the flash
read_flash_sfdp Read SPI flash SFDP (Serial Flash Discoverable Parameters)
merge_bin Merge multiple raw binary files into a single file for later flashing
get_security_info Get some security-related data
version Print esptool version

Get chip information:

uv tool run --from esptool esptool.py chip_id

Output:

esptool.py v4.8.1
Found 33 serial ports
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP8266
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: //(redacted)//
Uploading stub...
Running stub...
Stub running...
Chip ID: //(redacted)//
Hard resetting via RTS pin...

Install MicroPython

MicroPython is a software implementation of a programming language largely compatible with Python 3, written in C, that is optimized to run on a microcontroller.

MicroPython consists of a Python compiler to bytecode and a runtime interpreter of that bytecode. The user is presented with an interactive prompt (the REPL) to execute supported commands immediately. Included are a selection of core Python libraries; MicroPython includes modules which give the programmer access to low-level hardware.

from Wikipedia

Erase flash:

uv tool run --from esptool esptool.py erase_flash

Get the MicroPython firmware:

wget https://micropython.org/resources/firmware/ESP8266_GENERIC-20241129-v1.24.1.bin

Flash the firmware to the ESP8266:

uv tool run --from esptool esptool.py --baud 460800 write_flash --flash_size=detect 0 ESP8266_GENERIC-20241129-v1.24.1.bin

Test MicroPython

Install picocom (if needed) and connect to REPL:

sudo apt install picocom
picocom /dev/ttyUSB0 -b115200

Test the REPL:

>>> print('hello esp8266!')

hello esp8266!

Upload and Run a Python Script

Adapted from here: https://problemsolvingwithpython.com/12-MicroPython/12.06-Uploading-Code/

MicroPython Tool (ampy) is a utility to interact with a CircuitPython or MicroPython board over a serial connection.

Ampy is meant to be a simple command line tool to manipulate files and run code on a CircuitPython or MicroPython board over its serial connection. With ampy you can send files from your computer to the board’s file system, download files from a board to your computer, and even send a Python script to a board to be executed.

from PyPI

Install ampy:

uv add adafruit-ampy

Upload a script, then list the file contents of the microcontroller to confirm it’s there:

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 put hello.py

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 ls

Output:

/boot.py
/hello.py

Run the uploaded script:

uv tool run --from adafruit-ampy ampy -p /dev/ttyUSB0 run hello.py

Output:

Hello from esp-test!
]]> Python Embedded and IoT https://jcarr.codeberg.page/blog/posts/2025/esp8266_uv.html Sun, 05 Jan 2025 05:00:00 GMT Single-File Dependency Management in Python Script Using UV Jim Carr https://jcarr.codeberg.page/blog/posts/2025/single_file_depend_uv_python.html UV deserves a comprehensive overview post, but today I’m going to discuss a specific convenience feature that I love.

In the past, whenever I’ve had a Python script with a dependency that I want to manage in an isolated environment, I’ve set up a separate Bash script to bootstrap it by activating the virtual environment and then running the Python script. Ugly and bloated, but it works. With UV, however, you can streamline this by using inline dependency management.

Let’s walk through an example. (Make sure you already have uv installed.)

I don’t have Numpy installed globally, so we’ll use that in our example script. Create a file named numpy_version.py with the following contents:

import numpy as np

def show_numpy_version():
    print(np.__version__)

if __name__ == '__main__':
    show_numpy_version()

If we try to run it as-is (python3 numpy_version.py) we see this:

Traceback (most recent call last):
  File "numpy_version.py", line 1, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

We can use uv to add inline metadata declaring the dependencies:

uv add --script numpy_version.py numpy

The script will be updated to look like this:

# /// script
# requires-python = ">=3.12"
# dependencies = [
#     "numpy",
# ]
# ///
import numpy as np

def show_numpy_version():
    print(np.__version__)

if __name__ == '__main__':
    show_numpy_version()

Then, we use uv to run it:

uv run numpy_version.py

And we get this output:

Reading inline script metadata from `numpy_version.py`
2.2.1

We can make our script directly runnable by adding a shebang line at the top of the file:

#!/usr/bin/env -S uv run

…and making the script executable:

chmod u+x numpy_version.py

The -S argument to env is required in order for the run argument to be handled correctly.

Now we can run the script as ./numpy_version.py.

When we run the script like this, uv automatically manages the dependencies in an isolated environment. We can test this by trying to run the updated script directly with Python again:

python3 numpy_version.py

We get this, indicating that in a global context Numpy is still not installed:

Traceback (most recent call last):
  File "numpy_version.py", line 10, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

Our end result is a nice, clean, directly runnable, isolated script whose only execution requirement is that uv be installed.

The portability that this provides is really nice. I installed uv on a Le Potato and I was able to immediately run my script on the device with no changes. UV even automatically installed a CPython version to meet the Python version requirement. (The inline metadata specifies a Python version >= 3.12, but the global Python interpreter on the SBC is 3.11)

]]> Python https://jcarr.codeberg.page/blog/posts/2025/single_file_depend_uv_python.html Wed, 01 Jan 2025 05:00:00 GMT Total Dissolved Solids Measurements Using Arduino Jim Carr https://jcarr.codeberg.page/blog/posts/2024/tds_sensor_arduino.html Total dissolved solids (TDS) refer to the combined content of all inorganic and organic substances contained in a liquid, typically water, that are present in a molecular, ionized, or micro-granular form. TDS is usually measured in milligrams per liter (mg/L) or parts per million (ppm). (I’ll be using ppm in this article)

The components of TDS can include a variety of substances, such as:

  • Salts (e.g., sodium, calcium, magnesium, potassium)
  • Minerals (e.g., bicarbonates, sulfates, chlorides)
  • Organic matter (e.g., humic substances)

TDS is an important parameter in water quality assessment, as it can affect the physical and chemical properties of water, including its taste, clarity, and suitability for drinking, irrigation, and industrial processes. High levels of TDS can indicate water pollution or the presence of excessive minerals, which may be harmful to health or the environment.

Hardware

Software

You are welcome to use whichever software combination you prefer. However, I will be using the following tools, and the instructions in this article will be based on that setup:

Hardware Setup

Connect the TDS Sensor controller to the Arduino’s 5V, Ground, and A1 pins, and suspend the sensor probe in a container of water:

Source: CQRobot wiki

Code and Test

Use the Arduino CLI to create a sketch:

arduino-cli sketch new tds_arduino

cd tds_arduino

Create a new Makefile. This will simplify compiling, uploading, and monitoring our Arduino sketch:

CORE = arduino:avr
FQBN = $(CORE):uno
CLI_EXE = arduino-cli
PORT = /dev/ttyACM0
BAUD = 115200

default:
    @echo 'Targets:'
    @echo '  list'
    @echo '  compile'
    @echo '  upload'
    @echo '  monitor'

list:
    $(CLI_EXE) board list

compile:
    $(CLI_EXE) compile --fqbn $(FQBN) .

upload: compile
    $(CLI_EXE) upload -p $(PORT) --fqbn $(FQBN) .

monitor:
    $(CLI_EXE) monitor -p $(PORT) --config $(BAUD)

Update the contents of tds_arduino.ino as follows:

#define TdsSensorPin A1

void setup() {
  // Initialize serial communication at a baud rate of 115200 bits per second:
  Serial.begin(115200);

  // Set the mode of the pin to INPUT: This pin will be used to read data from
  // the TDS sensor:
  pinMode(TdsSensorPin, INPUT);
}

void loop() {
  static unsigned long analogSampleTimepoint = millis();

  // take a reading and display it every 1/2 second:
  if (millis() - analogSampleTimepoint > 500U) {
    analogSampleTimepoint = millis();

    // get sensor pin value
    // https://docs.arduino.cc/language-reference/en/functions/analog-io/analogRead/
    int sensorValue = analogRead(TdsSensorPin);

    Serial.print("Sensor value: ");
    Serial.println(sensorValue);
  }
}

Compile and upload the sketch, then run the monitor:

make upload

make monitor

Results:

Sensor value: 228
Sensor value: 228
Sensor value: 227
Sensor value: 228
Sensor value: 229
Sensor value: 229
Sensor value: 229
Sensor value: 228

The values we’re capturing at this point are not ppm of total dissolved solids: They’re voltage levels being reported from the sensor pin. Moreover, they aren’t actual voltage levels, but an abstraction of them. (The documentation for analogRead explains it well: https://docs.arduino.cc/language-reference/en/functions/analog-io/analogRead/)

In order to be useful, we need to convert them. First we’ll add some global values:

float referenceVoltage =
    5.0; // analog reference voltage of the analog-to-digital convertor (ADC)
float tdsValue = 0;

Next, our conversion function:

float getTdsValue(int sensorValue, float temperatureAtCollection) {
  // Convert the sensor pin reading to actual voltage:
  float sensorVoltage = (float)sensorValue * referenceVoltage / 1024.0;

  // temperature compensation formula: fFinalResult(25^C) =
  // fFinalResult(current)/(1.0+0.02*(fTP-25.0))
  float compensationCoefficient = 1.0 + 0.02 * (temperatureAtCollection - 25.0);

  // temperature compensation
  float compensationVoltage = sensorVoltage / compensationCoefficient;

  // convert voltage value to tds value
  float calculatedTdsValue =
      (133.42 * compensationVoltage * compensationVoltage *
           compensationVoltage -
       255.86 * compensationVoltage * compensationVoltage +
       857.39 * compensationVoltage) *
      0.5;

  return calculatedTdsValue;
}

The function takes in two values: the sensor reading and the temperature at the time of collection. (Since we aren’t working with a temperature sensor in this exercise, we’ll use a default value for that) The logic for voltage, temperature compensation, and voltage-to-tds come from the CQRobot wiki: http://www.cqrobot.wiki/index.php/TDS_(Total_Dissolved_Solids)_Meter_Sensor_SKU:_CQRSENTDS01#Arduino_Application

Also, the collection temperature must be in Celsius. Since we’re likely to be working with Fahrenheit, we’ll add a function for that conversion as well:

float fahrenheitToCelsius(float temperature) {
  float celsiusValue = ((temperature - 32) * 5) / 9;

  return celsiusValue;
}

Finally, we’ll update our processing loop to use the new functions:

void loop() {
  static unsigned long analogSampleTimepoint = millis();

  // take a reading and display it every 1/2 second:
  if (millis() - analogSampleTimepoint > 500U) {
    analogSampleTimepoint = millis();

    // get sensor pin value
    // https://docs.arduino.cc/language-reference/en/functions/analog-io/analogRead/
    int sensorValue = analogRead(TdsSensorPin);

    // Convert temperature to celsius, using a default temperature of 77 degrees
    // Fahrenheit:
    float celsiusValue = fahrenheitToCelsius(77);

    // Calculate the actual TDS value:
    tdsValue = getTdsValue(sensorValue, celsiusValue);

    Serial.print("TDS -- PPM: ");
    Serial.println(tdsValue);
  }
}

Now, our complete updated code looks like this:

#define TdsSensorPin A1

float referenceVoltage =
    5.0; // analog reference voltage of the analog-to-digital convertor (ADC)
float tdsValue = 0;

float fahrenheitToCelsius(float temperature) {
  float celsiusValue = ((temperature - 32) * 5) / 9;

  return celsiusValue;
}

float getTdsValue(int sensorValue, float temperatureAtCollection) {
  // Convert the sensor pin reading to actual voltage:
  float sensorVoltage = (float)sensorValue * referenceVoltage / 1024.0;

  // temperature compensation formula: fFinalResult(25^C) =
  // fFinalResult(current)/(1.0+0.02*(fTP-25.0))
  float compensationCoefficient = 1.0 + 0.02 * (temperatureAtCollection - 25.0);

  // temperature compensation
  float compensationVoltage = sensorVoltage / compensationCoefficient;

  // convert voltage value to tds value
  float calculatedTdsValue =
      (133.42 * compensationVoltage * compensationVoltage *
           compensationVoltage -
       255.86 * compensationVoltage * compensationVoltage +
       857.39 * compensationVoltage) *
      0.5;

  return calculatedTdsValue;
}

void setup() {
  // Initialize serial communication at a baud rate of  115200 bits per second:
  Serial.begin(115200);

  // Set the mode of the pin to INPUT: This pin will be used to read data from
  // the TDS sensor:
  pinMode(TdsSensorPin, INPUT);
}

void loop() {
  static unsigned long analogSampleTimepoint = millis();

  // take a reading and display it every 1/2 second:
  if (millis() - analogSampleTimepoint > 500U) {
    analogSampleTimepoint = millis();

    // get sensor pin value
    // https://docs.arduino.cc/language-reference/en/functions/analog-io/analogRead/
    int sensorValue = analogRead(TdsSensorPin);

    // Convert temperature to celsius, using a default temperature of 77 degrees
    // Fahrenheit:
    float celsiusValue = fahrenheitToCelsius(77);

    // Calculate the actual TDS value:
    tdsValue = getTdsValue(sensorValue, celsiusValue);

    Serial.print("TDS -- PPM: ");
    Serial.println(tdsValue);
  }
}

When we compile, upload, and run our serial monitor, we now get the PPM values we want:

make upload

make monitor
TDS -- PPM: 403.13
TDS -- PPM: 405.03
TDS -- PPM: 403.13
TDS -- PPM: 403.13
TDS -- PPM: 403.13
TDS -- PPM: 405.03
TDS -- PPM: 403.13
]]> Embedded and IoT https://jcarr.codeberg.page/blog/posts/2024/tds_sensor_arduino.html Sun, 17 Nov 2024 05:00:00 GMT Observing Plan for the November Astronomy Program at Garber Forest Jim Carr https://jcarr.codeberg.page/blog/posts/2024/garber_observing_plan.html

IMPORTANT UPDATE! Due to the forecast of clouds and showers on Saturday evening, I regret to inform you that I must cancel. So sorry! I’m hoping that I can re-schedule next Spring.

On November 9th, 2024, I will be at the Garber Forest near Lewisburg Ohio, equipped with my telescope to discuss astronomy and lead an exploration of the night sky - weather permitting, of course!

This event is organized by the Preble County Park District.

What We’ll Look For

8pm

Jupiter, Aldebaran, Pleiades, Capella

To the North-East-East, Jupiter has just risen and is about 8 degrees above the horizon, shining with a magnitude of -2.74 between the constellations of Auriga (“the charioteer”) and Taurus (“the bull”). It’s pretty low in the sky for observation, so we’ll come back to it later.

To the right of Jupiter is the bright star Aldebaran (the “eye” of Taurus) and directly above it (about 26 degrees above the horizon) is the Pleiades star cluster.

To the North-East, at about the same altitude as the Pleiades, is the bright star Capella, part of Auriga.

Moon and Saturn

Looking to the South, a waxing Moon just over half full shines between the constellations Aquarius (“the water-carrier”) and Capricornus (“the horned goat”), at about 33 degrees above the horizon. Just up and to the left of the moon is Saturn (magnitude 0.85), in Aquarius. Looking closely, the rings of Saturn are almost edge-on, but hopefully we’ll be able to make them out.

The Summer Triangle

High in the sky to the west we see the three bright stars Altair, Deneb, and Vega. Although these stars are in separate constellations (Aquila, Lyra, and Cygnus, respectively) together they make up “The Summer Triangle”, a well-known asterism.

Ursa Major, Ursa Minor, Polaris

Finally, to the North, Ursa Major (“the big dipper”) sits low on the horizon and above it is Ursa Minor (“the little dipper”) with Polaris (the steadfast “North Star”).

9pm

Jupiter will now be about 19 degrees above the horizon. Looking at it through the telescope, we should be able to make out its three brightest moons: Callisto, Europa, and Ganymede. As we get closer to 10pm, the moon Io may become visible for us as well.

10pm

Orion

The constellation Orion (“the hunter”) will have risen, but will still be low in the East. It’s easy to spot, with its bright stars Betelgeuse, Rigel, and Bellatrix, and the three distinct stars of Orion’s belt: Mintaka, Alnilam, and Alnitak. Will it be high enough for us to see? If so, we might also catch a glimpse of the Orion Nebula, the “sword” hanging from Orion’s belt.

More Information

Magnitude

In astronomy, “magnitude” refers to the brightness of celestial objects, such as stars, planets, and galaxies. There are two main types of magnitude:

  1. Apparent Magnitude: This measures how bright an object appears from Earth. The scale is logarithmic and inversely related, meaning that lower numbers indicate brighter objects. For example, a star with an apparent magnitude of 1 is brighter than a star with an apparent magnitude of 6. The scale also includes negative values for very bright objects, such as the Sun and the Moon.

  2. Absolute Magnitude: This measures the intrinsic brightness of a celestial object, defined as how bright the object would appear if it were located at a standard distance of 10 parsecs (about 32.6 light-years) from Earth. Absolute magnitude allows astronomers to compare the true brightness of different objects without the effects of distance.

The concept of magnitude is crucial for understanding the luminosity and distance of celestial bodies, as well as for classifying stars and other astronomical phenomena.

Constellation

A constellation is a recognized pattern of stars in the night sky that forms a specific shape or figure, often representing mythological characters, animals, or objects. Constellations serve as a way to organize and identify groups of stars, making it easier for astronomers and stargazers to navigate the sky.

There are 88 officially recognized constellations, as defined by the International Astronomical Union (IAU). These constellations cover the entire celestial sphere and are used in various ways, including for navigation, storytelling, and as a framework for locating celestial objects. Some well-known constellations include Orion, Ursa Major, and Cassiopeia.

Asterism

In astronomy, “asterism” refers to a recognizable pattern or group of stars that may not necessarily form a formal constellation. Asterisms can be made up of stars from one or more constellations and are often more easily recognizable or familiar to the casual observer.

For example, the Big Dipper is an asterism that is part of the larger constellation Ursa Major. Similarly, the Summer Triangle is another well-known asterism formed by three bright stars from three different constellations: Vega (Lyra), Deneb (Cygnus), and Altair (Aquila).

Asterisms are useful for amateur astronomers and stargazers as they can serve as reference points for locating other stars and celestial objects in the night sky.

Altitude and Azimuth

Altitude and azimuth are two coordinates used in astronomy to specify the position of an object in the sky as observed from a specific location on Earth.

Altitude and Azimuth

  1. Altitude: This is the angle between the object in the sky and the observer’s local horizon. It is measured in degrees, ranging from 0° (the horizon) to 90° (the zenith, which is directly overhead). An object with an altitude of 0° is on the horizon, while an object with an altitude of 90° is directly above the observer.

  2. Azimuth: This is the angle measured clockwise from a reference direction, typically true north. Azimuth is also measured in degrees, ranging from 0° to 360°. For example, an azimuth of 0° corresponds to north, 90° to east, 180° to south, and 270° to west.

Together, altitude and azimuth provide a way to pinpoint the location of celestial objects in the sky relative to an observer’s position on Earth. This system is particularly useful for navigation, stargazing, and astronomical observations.

The Moon

The Moon is Earth’s only natural satellite and is the fifth largest moon in the solar system. It has a significant impact on Earth, influencing tides and stabilizing the planet’s axial tilt. The Moon’s surface is covered with craters, mountains, and flat plains called maria, which were formed by ancient volcanic activity.

The Moon has a synchronous rotation, meaning it rotates on its axis in the same time it takes to orbit Earth, which is why we always see the same side from our planet. The far side of the Moon, often mistakenly called the “dark side,” is not permanently dark; it just faces away from Earth.

The Moon has been a subject of human fascination for centuries, inspiring myths, art, and scientific exploration. The Apollo missions in the late 1960s and early 1970s marked the first time humans set foot on the Moon, with Apollo 11 being the most famous mission, landing Neil Armstrong and Buzz Aldrin on its surface in 1969.

Moon Phases

Waxing and waning refer to the phases of the moon as it orbits the Earth, specifically how the visible portion of the moon changes over time.

  1. Waxing: This term describes the period when the moon is increasing in illumination. After the new moon phase, the moon begins to wax, meaning more of its surface becomes visible from Earth. The phases during this period include:
    • Waxing Crescent: A small sliver of the moon is illuminated.
    • First Quarter: Half of the moon is illuminated.
    • Waxing Gibbous: More than half of the moon is illuminated but not yet full.
  2. Waning: This term describes the period when the moon is decreasing in illumination. After the full moon phase, the moon begins to wane, meaning less of its surface is visible. The phases during this period include:
    • Waning Gibbous: More than half of the moon is illuminated but decreasing.
    • Last Quarter: Half of the moon is illuminated, but the opposite side compared to the first quarter.
    • Waning Crescent: A small sliver of the moon is illuminated, leading back to the new moon phase.

These cycles occur roughly every 29.5 days, which is the time it takes for the moon to complete one full orbit around the Earth and return to the same phase.

Planets

There are eight recognized planets in the solar system, which are divided into two categories:

  • Terrestrial planets: Mercury, Venus, Earth, and Mars. These are rocky and have solid surfaces.
  • Gas giants: Jupiter and Saturn, which are primarily composed of hydrogen and helium.
  • Ice giants: Uranus and Neptune, which have a larger proportion of “ices” such as water, ammonia, and methane.

Dwarf Planets: These include Pluto, Eris, Haumea, Makemake, and Ceres. Dwarf planets are similar to regular planets but do not clear their orbital paths of other debris.

Jupiter

Jupiter is the largest planet in our solar system, known for its massive size, strong magnetic field, and distinctive features. Here are some key points about Jupiter:

  1. Size and Composition: Jupiter is a gas giant, primarily composed of hydrogen and helium. It has a diameter of about 86,881 miles (139,822 kilometers), making it more than 11 times wider than Earth.

  2. Great Red Spot: One of Jupiter’s most famous features is the Great Red Spot, a giant storm that has been raging for at least 350 years. It is larger than Earth and is characterized by its reddish color.

  3. Moons: Jupiter has a large number of moons, with 79 confirmed as of now. The four largest moons, known as the Galilean moons, are Io, Europa, Ganymede, and Callisto. Ganymede is the largest moon in the solar system.

  4. Rings: Jupiter has a faint ring system composed mainly of dust particles and small rocks. The rings are not as prominent as those of Saturn.

  5. Magnetic Field: Jupiter has the strongest magnetic field of any planet in the solar system, which is generated by the motion of metallic hydrogen in its interior.

  6. Exploration: Several spacecraft have visited Jupiter, including the Galileo orbiter and the Juno spacecraft, which is currently studying the planet’s atmosphere, magnetic field, and structure.

  7. Atmosphere: Jupiter’s atmosphere is known for its colorful bands and storms, including the aforementioned Great Red Spot. The planet’s atmosphere is dynamic and experiences high-speed winds.

Jupiter’s Moons

Callisto

Callisto is one of the largest moons of Jupiter and is the third-largest moon in the solar system. It has a diameter of about 4,820 kilometers (2,995 miles) and is known for its heavily cratered surface, which suggests it has been geologically inactive for a long time. Callisto is composed primarily of ice and rock, and its surface features include a mix of old impact craters and some younger, less cratered regions.

One of the most interesting aspects of Callisto is its potential subsurface ocean. While there is no direct evidence of this ocean, some scientists believe that beneath its icy crust, there may be a salty ocean that could harbor conditions suitable for life.

Callisto has a very thin atmosphere, primarily composed of carbon dioxide, and it has a low level of radiation compared to other Galilean moons, making it a more favorable location for future exploration and potential human missions. It was discovered by Galileo Galilei in 1610, along with the other three largest moons of Jupiter: Io, Europa, and Ganymede.

Europa

Europa is one of Jupiter’s largest moons and is the sixth-largest moon in the solar system. It is particularly interesting to scientists because it is believed to have a subsurface ocean beneath its icy crust, which could potentially harbor conditions suitable for life. Here are some key features of Europa:

  1. Surface and Composition: Europa’s surface is primarily composed of water ice, and it features a smooth, bright appearance with few impact craters, suggesting a relatively young surface. The cracks and ridges on its surface indicate tectonic activity.

  2. Subsurface Ocean: Scientists believe that beneath Europa’s icy shell lies a salty ocean that may be in contact with the moon’s rocky mantle. This ocean could provide the necessary chemical ingredients for life.

  3. Potential for Life: The combination of liquid water, a stable energy source (from tidal heating due to Jupiter’s gravitational pull), and essential chemical elements makes Europa a prime candidate in the search for extraterrestrial life.

  4. Exploration: Europa has been studied by several spacecraft, including the Galileo orbiter and the Hubble Space Telescope.

  5. Atmosphere: Europa has a very thin atmosphere composed mainly of oxygen, but it is far too thin to support human life.

Europa continues to be a focus of astrobiological research and exploration due to its intriguing characteristics and the possibility of finding life beyond Earth.

Europa Clipper, a space probe developed by NASA, is currently on its way to Europa and is expected to arrive in the Jupiter system in 2030.

The goals of Europa Clipper are to explore Europa, investigate its habitability, and aid in the selection of a landing site for the proposed Europa Lander. This exploration is focused on understanding the three main requirements for life: liquid water, chemistry, and energy. Specifically, the objectives are to study:

  • Ice shell and ocean: Confirm the existence and characterize the nature of water within or beneath the ice, and study processes of surface-ice-ocean exchange.
  • Composition: Distribution and chemistry of key compounds and the links to ocean composition.
  • Geology: Characteristics and formation of surface features, including sites of recent or current activity.

source

Ganymede

Ganymede is the largest moon of Jupiter and the largest moon in the solar system. It is one of the four Galilean moons, which were discovered by Galileo Galilei in 1610. Ganymede has a diameter of about 5,268 kilometers (3,273 miles), making it even larger than the planet Mercury, although it does not have enough mass to be classified as a planet.

Ganymede is unique among moons because it has a magnetic field, which is likely generated by a liquid iron or iron-sulfide core. The surface of Ganymede is a mix of two types of terrain: bright, icy regions and darker, heavily cratered areas. It is believed to have a subsurface ocean beneath its icy crust, which raises the possibility of conditions that could support life.

In addition to its geological features, Ganymede has a thin atmosphere composed mostly of oxygen, although it is far too thin to support human life. The moon has been studied by various spacecraft, including the Galileo orbiter and the Hubble Space Telescope.

Io

Io is one of the four largest moons of Jupiter. It is the most geologically active body in the solar system, with hundreds of volcanoes, some of which are still erupting. Io’s surface is covered with sulfur and sulfur dioxide, giving it a colorful appearance with shades of yellow, red, and white.

The intense geological activity on Io is primarily due to the gravitational pull from Jupiter and the other Galilean moons (Europa, Ganymede, and Callisto), which creates tidal heating. This process generates enough internal heat to drive volcanic activity.

Io has a thin atmosphere composed mainly of sulfur dioxide, and its surface temperature can vary widely. The moon also has a very weak magnetic field and is known to have a significant interaction with Jupiter’s magnetosphere.

Saturn

Saturn is the sixth planet from the Sun and is known for its prominent ring system, which is the most extensive and complex in the solar system. It is a gas giant, primarily composed of hydrogen and helium, and has a diameter of about 86,881 miles (139,822 kilometers). Saturn has at least 82 moons, with Titan being the largest and one of the most interesting due to its thick atmosphere and surface lakes of liquid methane and ethane.

Saturn’s rings are made up of ice particles, rocky debris, and dust, and they vary in thickness and density. The planet has a relatively low density, meaning it could float in water if there were a body of water large enough to contain it. Saturn is also known for its strong winds, which can reach speeds of over 1,100 miles per hour (1,800 kilometers per hour) in its upper atmosphere.

Bright Stars

Aldebaran

Aldebaran is a prominent star located in the constellation Taurus. It is often referred to as the “Eye of the Bull” and is one of the brightest stars in the night sky. Aldebaran is classified as a red giant star and is approximately 65 light-years away from Earth. It has a distinct orange hue and is part of the Hyades star cluster, although it is not physically part of the cluster itself. Aldebaran is notable for its brightness and is often used in navigation and astronomy.

Altair

Altair is a prominent star located in the constellation Aquila, which represents an eagle. Here are some key facts about Altair:

  1. Brightness: Altair is one of the brightest stars in the night sky and is the 12th brightest star overall. It has an apparent magnitude of about 0.76.

  2. Distance: Altair is approximately 16.7 light-years away from Earth, making it one of the closest stars to our solar system.

  3. Spectral Type: Altair is classified as an A7V main-sequence star, which means it is a hot, white star that is in the stable phase of its life cycle, fusing hydrogen into helium in its core.

  4. Rotation: Altair is known for its rapid rotation, completing a full rotation approximately every 9 hours. This fast rotation causes it to have an oblate shape, meaning it is flattened at the poles and bulging at the equator.

  5. Cultural Significance: In various cultures, Altair has been associated with different myths and stories. In Chinese astronomy, it is known as “Zhi Nu,” the Weaving Girl, and is part of the famous “Cowherd and Weaving Girl” legend.

  6. Part of the Summer Triangle: Altair is one of the three stars that form the Summer Triangle, along with Vega (in Lyra) and Deneb (in Cygnus). This asterism is prominent in the summer sky in the Northern Hemisphere.

Bellatrix

Bellatrix, also known as Gamma Orionis, is a prominent star located in the constellation Orion. Here are some key details about Bellatrix:

  1. Brightness: Bellatrix is one of the brightest stars in the night sky and is classified as a blue giant. It has an apparent magnitude of about 1.64, making it the third-brightest star in Orion, after Betelgeuse and Rigel.

  2. Distance: Bellatrix is approximately 240 light-years away from Earth.

  3. Spectral Type: It is classified as a B2 III star, indicating that it is a blue giant with a surface temperature of around 22,000 Kelvin.

  4. Characteristics: Bellatrix is known for its rapid rotation and is significantly more massive than the Sun. It is also in a later stage of stellar evolution compared to many other stars, having exhausted the hydrogen in its core and expanded in size.

  5. Name Origin: The name “Bellatrix” comes from the Latin word for “female warrior,” which is fitting given its position in the constellation representing a hunter.

Bellatrix is often used in astronomy and stargazing due to its brightness and position in the well-known Orion constellation.

Betelgeuse

Betelgeuse is a red supergiant star located in the constellation Orion. It is one of the largest and most luminous stars known, with a diameter estimated to be about 1,000 times that of the Sun. Betelgeuse is notable for its distinct reddish color and is often used as a reference point in the night sky.

As a supergiant, Betelgeuse is in the later stages of its stellar evolution and is expected to eventually explode as a supernova. This event could happen within the next million years, although it’s difficult to predict exactly when.

In addition to its astronomical significance, Betelgeuse has captured the imagination of many cultures and is often featured in literature and popular media.

Betelgeuse was in the news recently.

Capella

Capella is one of the brightest stars in the night sky and is located in the constellation Auriga. Here are some key facts about Capella:

  1. Brightness: Capella is the sixth-brightest star in the night sky and is classified as a bright giant star.

  2. Distance: It is approximately 42 light-years away from Earth.

  3. Binary System: Capella is actually a binary star system, consisting of two giant stars, Capella Aa and Capella Ab, which are both yellowish in color. They are orbiting each other and are surrounded by a faint third component, Capella C, which is a much fainter red dwarf star.

  4. Spectral Type: The primary stars (Capella Aa and Ab) are classified as G-type stars, similar to our Sun, but they are larger and more luminous.

  5. Cultural Significance: Capella has been known since ancient times and has been referenced in various cultures and mythologies. In Latin, “Capella” means “little goat,” which is a reference to its association with the goat in the constellation.

Capella is often visible in the evening sky during the winter months in the Northern Hemisphere.

Deneb

Deneb is one of the brightest stars in the night sky and is part of the constellation Cygnus, also known as the Swan. It is located approximately 1,425 light-years away from Earth and is one of the vertices of the Summer Triangle, along with the stars Vega and Altair. Deneb is a supergiant star, classified as a spectral type A2 Ia, and is known for its significant luminosity and size. It is also notable for its role in various cultural and astronomical contexts, often associated with navigation and mythology.

Polaris

“Polaris,” also known as the North Star, is the brightest star in the constellation Ursa Minor. It is located nearly directly above the North Pole, making it a crucial point of reference for navigation in the Northern Hemisphere. Polaris is a supergiant star and is approximately 433 light-years away from Earth.

Rigel

Rigel is a prominent star located in the constellation Orion. It is one of the brightest stars in the night sky and is classified as a blue supergiant. Rigel is approximately 860 light-years away from Earth and has a surface temperature of around 12,000 Kelvin, which gives it its blue color. It is often used in navigation and is notable for its brightness and distinctive position in the Orion constellation.

Vega

Vega is one of the brightest stars in the night sky and is located in the constellation Lyra. Here are some key facts about Vega:

  1. Brightness: Vega is the fifth-brightest star in the night sky and is one of the three stars that form the Summer Triangle, along with Deneb and Altair.

  2. Distance: It is approximately 25 light-years away from Earth.

  3. Type: Vega is classified as an A-type main-sequence star (spectral type A0V). It is about twice as massive as the Sun and has a surface temperature of around 9,600 K, which gives it a bluish-white color.

  4. Rotation: Vega has a rapid rotation speed, completing a rotation in about 12.5 hours, which is relatively fast compared to the Sun.

  5. Historical Significance: Vega has been used as a reference point for the calibration of the brightness of other stars. It was also the first star other than the Sun to be photographed and has been studied extensively in various fields of astronomy.

  6. Cultural References: Vega has been significant in various cultures and mythologies. In Chinese mythology, it is associated with the weaver girl, and in Western culture, it has been referred to as the “Harp Star.”

Deep Sky Objects

Orion Nebula

The Orion Nebula, also known as M42, is one of the most famous and studied nebulae in the night sky. It is located in the Milky Way, approximately 1,344 light-years away from Earth in the Orion constellation. The nebula is a stellar nursery, where new stars are being born from the surrounding gas and dust.

Key features of the Orion Nebula include:

  1. Brightness: It is one of the brightest nebulae visible to the naked eye and can be seen as a fuzzy patch in the sword of Orion.

  2. Size: The Orion Nebula spans about 24 light-years across and contains a vast amount of gas and dust.

  3. Star Formation: The nebula is home to a young cluster of stars known as the Trapezium, which consists of four massive stars that illuminate the surrounding gas and dust.

  4. Observation: The Orion Nebula is a popular target for amateur astronomers and astrophotographers due to its brightness and the intricate structures within it.

  5. Scientific Importance: It provides valuable insights into the processes of star formation and the dynamics of interstellar matter.

The Orion Nebula is a stunning example of the beauty and complexity of the universe, making it a favorite subject for both professional and amateur astronomers.

The Pleiades

The Pleiades, also known as the Seven Sisters, is a star cluster located in the constellation Taurus. It is one of the nearest star clusters to Earth and is easily visible to the naked eye. The cluster contains several hundred stars, but the seven brightest are typically the ones referred to as the “Seven Sisters.” These stars are named Alcyone, Asterope, Electra, Maia, Merope, Taygeta, and Pleione.

The Pleiades has been significant in various cultures throughout history, often associated with mythology and folklore. In astronomy, it is known for its beauty and is often used as a reference point for navigation. The cluster is also of interest to astronomers studying star formation and the evolution of stars.

]]> Astronomy https://jcarr.codeberg.page/blog/posts/2024/garber_observing_plan.html Fri, 18 Oct 2024 04:00:00 GMT Detecting the Voyager Spacecraft in Green Bank Telescope Data Jim Carr https://jcarr.codeberg.page/blog/posts/2024/voyager_python.html

Breakthrough Listen

Breakthrough Listen is a project to search for intelligent extraterrestial communications from deep space. The project uses radio wave observations from the Green Bank Observatory and the Parkes Observatory, and visible light observations from the Automated Planet Finder. In this article, I’ll be focusing on radio data collected from the Green Bank Observatory.

Observing Strategy

The basic observing strategy for GBT data is:

  • Observe a target (“star A”) from the primary target list for 5 minutes.
  • Observe another target (“star B”) at a nearby position on the sky for 5 minutes.
  • Return to star A for 5 minutes.
  • Observe another target (“star C”) from the secondary list for 5 minutes.
  • Another 5 minutes on star A.
  • 5 minutes on another secondary target (“star D”)

This ABACAD cadence is repeated for all of the “A” stars in the primary target list (around 1700 in total, although not all of these are visible from the Green Bank Telescope site).

A signal would be interesting from a SETI standpoint if it’s present in all three “A” (or ON primary source) observations, but absent in the “B/C/D” (or OFF primary source) observations. If someone was standing near the telescope with a cellphone switched on, we wouldn’t expect this interference to come and go as we move on and off the primary target on the sky, so this is a good way of ruling out the local interference that makes up the majority of RFI.

source

Challenges

There are multiple challenges to contend with in the search for these signals, including:

  • Human technology gives off signals like the ones being searched for (radio frequency interference)
  • Data volumes are too large to run brute force analysis on entire datasets.
  • The sky is big: There are many targets to evaluate.

But, there’s another challenge:

How do we calibrate instruments, and test our detection methods?

What we really need is a known faint signal from space. Luckily, we have one.

Voyager 1

Voyager 1 is a space probe launched by NASA on September 5, 1977, as part of the Voyager program, aimed at exploring the outer Solar System and the interstellar space beyond the Sun’s heliosphere. It was launched 16 days after its twin, Voyager 2. The probe communicates with Earth via the NASA Deep Space Network (DSN) to receive commands and send data back. NASA and JPL provide real-time information on its distance and velocity. As of August 2024, Voyager 1 is located 164.0 AU (24.5 billion km; 15.2 billion mi) from Earth, making it the most distant human-made object in existence. During its mission, Voyager 1 conducted flybys of Jupiter, Saturn, and Saturn’s largest moon, Titan. NASA opted for a flyby of Titan over Pluto due to Titan’s significant atmosphere. The probe gathered valuable data on the weather, magnetic fields, and rings of the two gas giants and was the first to capture detailed images of their moons.

After 38 years, Voyager 1 continues to transmit telemetry data from the far reaches of interstellar space. This provides an excellent opportunity to test the Breakthrough Listen signal processing pipeline.

GBT Data

On December 30, 2015, the Greenbank X-band receiver (8.0-11.6 GHz) was utilized to observe the known location of Voyager 1. The BL digital signal processing system captures digitized data in a ‘raw’ format, which is then converted into ‘filterbank’ format - a straightforward binary file format described in the SIGPROC user guide.

In this tutorial, we’ll walk through the setup and creation of a Jupyter notebook to load and interpret the filterbank file.

A more detailed version of this tutorial, featuring extra scientific information, can be found here. My aim in recreating this tutorial is to provide additional insights into the environment setup.

Environment

This assumes a Linux environment, but it shouldn’t be difficult to adapt to others, if needed.

Visual Studio Code and Python/Jupyter

We’ll be using Jupyter notebooks, so we need an environment to run them in. I use Visual Studio Code with the Python extension, as it provides a very easy setup:

  1. Install Visual Studio Code.
  2. Open Visual Studio Code and install the Python extension.

The Python extension includes comprehensive support for Jupyter notebooks.

Python Virtual Environment

We have very specific scientific library requirements, so we’ll isolate our setup in a virtual environment.

Open a terminal.

Create a virtual environment:

python3 -m venv voyager_venv

Switch to the environment directory and activate it:

cd voyager_venv/

source bin/activate

Install Python dependencies for a scientific environment:

pip install numpy

pip install scipy

pip install matplotlib

pip install astropy

You’ll also need Blimpy:

pip install blimpy

Source Prep

Create a src/ directory to hold our notebook and associated data:

mkdir src/

cd src/

Download the filterbank file that includes the Voyager data:

wget https://storage.googleapis.com/gbt_fil/voyager_f1032192_t300_v2.fil

Open the directory in Visual Studio Code:

code .

Create the Notebook

Create a new file named voyager.ipynb. VS Code will initialize it as a Jupyter notebook.

On the right side of the notebook interface you’ll see link named “Select Kernel”. Click it, select “Python Environments”, then select the bin/python inside your virtual environment. You’ll end up with something that looks like this:

Let’s make sure our Python environment is working correctly. Inside the empty cell at the top of notebook, type this:

print("OK!")

Then, click “Run All”. Since this is your first time executing Python code in a notebook in the environment, you may be prompted to install the ipykernel package. If so, go ahead and click “Install”.

After the kernel installs, the notebook will execute and you’ll see this:

Extract the Data

Now that we’ve confirmed that our Python/Jupyter setup is working correctly, we can start the process of working with the Voyager data.

We want our plots to display inline in the notebook, so change the first cell to this:

%matplotlib inline

Add a new code block with the following imports:

import pylab as plt

from blimpy import Waterfall
  • pylab is a convenience module that combines matplotlib’s plotting functions with numpy’s numerical functions.
  • blimpy is a Python package used for working with radio astronomy data, particularly for handling and analyzing data from radio telescopes. The Waterfall class is typically used to represent and manipulate waterfall plots, which are a common way to visualize time-frequency data.

Add another cell and initialize the plot size:

plt.figure(figsize=(10,6))

Add another cell with the following:

obs = Waterfall('voyager_f1032192_t300_v2.fil')

This loads the Voyager observation data from the filterbank file.

Add another cell and get some information from the filterbank header:

obs.info()

The output will look something like this:

The specifics are discussed in detail in the SIGPROC user guide. Briefly, astronomers use a Celestial coordinate system to specify the location of objects in outer space. The src_raj and src_dej specify the J2000 coordinates, in terms of Right Ascension and Declination (RA & DEC), toward which the telescope is pointing. tstart specifies the Julian Date of the observation, and fch1 and foff specify the frequency of the first and frequency increment of each data channel respectively, in MHz.

source

Add another cell and plot the observation data’s spectrum:

obs.plot_spectrum()

What you’re seeing is the power spectral density of the data contained in the filterbank file. There are discontinuities due to the BL digital filters, but not a lot else. Voyager is a very narrowband signal, so we can’t easily see it in this plot. In order to do that, we need to zoom in. We use f_start and f_stop arguments to specify the beginning and ending frequency of the slice we want to view:

obs.plot_spectrum(f_start=8420.18, f_stop=8420.25)

There it is: the telemetry signal from Voyager 1! What you see is the carrier (center) and two sidebands that carry the data.

What Have We Accomplished?

In 2015, Green Bank pointed their radio telescope in the direction of a tiny spacecraft, billions of miles away, emitting a very faint signal. They captured a broad swath of radio frequency spectrum into a raw data file.

That file was converted into filterbank, a more accessible format. We then used simple software tools to reach into that data and pull out a very thin slice of it, bringing the Voyager signal into view.

]]> Python Astronomy https://jcarr.codeberg.page/blog/posts/2024/voyager_python.html Sun, 08 Sep 2024 04:00:00 GMT Modern Perl Jim Carr https://jcarr.codeberg.page/blog/posts/2024/modern_perl.html Last year, I wrote an article detailing my experience re-creating a Python script in Perl. This was a pretty cumbersome exercise, but I didn’t use many of Perl’s newer features. I decided to re-visit this, leveraging some of these newer features to see how well it improves the process.

Enabling Features

To take advantage of specific features, use the aptly named feature pragma.

You can enable an individual feature by name, and multiple features at once with qw(). For example, this:

use feature 'fc';
use feature 'say';

Is the same as this:

use feature qw(fc say);

Full example:

use feature qw(fc say);

$x = 'Test';

say "The case-folded version of $x is: " . fc $x;

Output:

The case-folded version of Test is: test

Without use feature, you’d get this:

String found where operator expected (Do you need to predeclare "say"?) at ./test.pl line 7, near "say "The case-folded version of $x is: ""
syntax error at ./test.pl line 7, near "say "The case-folded version of $x is: ""
Execution of ./test.pl aborted due to compilation errors.

You can also take advantage of feature bundles, enabling all of the features available as of a given version:

# implicitly loads 5.36 feature bundle

use v5.36;

But, you must also have (at least) that version of Perl installed, of course:

use v5.40;
Perl v5.40.0 required--this is only v5.38.2, stopped at ./test.pl line 3.
BEGIN failed--compilation aborted at ./test.pl line 3.

Useful Features

Lots of new features have been added in recent years, but I’ll take advantage of only a small subset of them for this exercise. I’ll be using features available in 5.38.2 and earlier.

Feature Version Notes
New class feature 5.38.0 Adds support for object-oriented code.
New say built-in 5.10.0 Works like print, adds a newline.
Subroutine signatures no longer considered experimental 5.36.0 Supports named parameters in subroutines.

Refactoring As Structured Code

We’ll start with the first subroutine that’s called.

Here’s the original:

sub exec_backup_set {
    my @source_paths = @{ [0] };
    my $target_path  = [1];

    foreach my $source_path (@source_paths) {
        exec_backup( $source_path, $target_path );
    }
}

And here’s the new version, using the signatures feature to implement subroutine arguments as lexical variables:

sub exec_backup_set ( $target_path, @source_paths ) {
    foreach my $source_path (@source_paths) {
        exec_backup( $source_path, $target_path );
    }
}

You’ll notice that using named arguments in the method signature is a lot cleaner. But, we have to swap the source_paths and target_path arguments. Why? Since source_paths is an array, it’s a ‘slurpy’ argument, meaning it will reference all remaining arguments in the signature. So, if it were the first argument, it would also absorb the contents of target_path.

A “slurpy” parameter is a list or hash parameter that “slurps up” all remaining arguments. Since any following parameters can’t receive values, there can be only one slurpy parameter.

Slurpy parameters must come at the end of the signature and they must be positional.

Slurpy parameters are optional by default.

https://metacpan.org/pod/Method::Signatures#Slurpy-parameters

Next, we’ll refactor the exec_backup subroutine. Here’s the original:

sub exec_backup {
    my ( $source, $target ) = @_;

    my $proc_name = "rsync -lrtv --delete \"${source}\" \"${target}\"";

    unless ( -d $target ) {
        mkdir($target);
    }

    if ( -d $target ) {
        print("Syncing ${source}...\n");
        system($proc_name);
        print("Synced ${source}\n");
    }

    print("----------\n");
}

For our updated version, we’ll use the signatures feature again, along with the say feature:

sub exec_backup_modern ( $source, $target ) {
    my $proc_name = "rsync -lrtv --delete \"${source}\" \"${target}\"";

    unless ( -d $target ) {
        mkdir($target);
    }

    if ( -d $target ) {
        say("Syncing ${source}...");

        system($proc_name);
        
        say("Synced ${source}");
    }

    say("----------");
}

Now we have enough information to refactor our entire script. Here’s the “modern” version of our old script:

use strict;
use warnings;

use feature 'signatures';
use feature 'say';
 
sub exec_backup ( $source, $target ) {
    my $proc_name = "rsync -lrtv --delete \"${source}\" \"${target}\"";

    unless ( -d $target ) {
        mkdir($target);
    }

    if ( -d $target ) {
        say("Syncing ${source}...");

        system($proc_name);

        say("Synced ${source}");
    }

    say("----------");
}

sub exec_backup_set ( $target_path, @source_paths ) {
    foreach my $source_path (@source_paths) {
        exec_backup( $source_path, $target_path );
    }
}
 
my @target_paths =
  ( "/target1/", "/target2/" );
 
# regular file sets
my @regular_files = (
    "/home/jimc/source1",  "/home/jimc/source2",
    "/home/jimc/source3",  "/home/jimc/source4"
);
exec_backup_set( $target_paths[0], @regular_files );
 
# large file sets
my @large_files = ( "/home/jimc/large1", "/home/jimc/large2" );
exec_backup_set( $target_paths[1], @large_files );

sleep(2);  # pause before exiting

With these changes, we have a script that’s more concise and readable.

Refactoring As Object-Oriented Code

We’ve improved the code, but what if we want to implement it using an object-oriented paradigm instead of structured? We can accomplish this using the class feature.

Let’s start with a very simple example:

use strict;
use warnings;

use feature 'class';
use feature 'say';

class MyUtil::Backup {

    method say_hello {
        say("Hello!");
    }
}

my $backup_obj = MyUtil::Backup->new();
$backup_obj->say_hello;

If you’re familiar with “old” Perl, this will look very unusual. We have a couple of new keywords we’re using, class and method. In this example, we’ve defined a new class Backup in a namespace MyUtil. Inside the class, we have a single callable method named say_hello. We create an instance of the class named $backup_obj and then call the say_hello method. We run the code and…

class is experimental at ./fullsync_oop.pl line 9.
method is experimental at ./fullsync_oop.pl line 11.
Hello!

The code runs, but the interpreter is warning us about the experimental nature of the class and method keywords. We can suppress this with a no warnings directive. With our code now looking like this:

use strict;
use warnings;

use feature 'class';
use feature 'say';

no warnings 'experimental::class';

class MyUtil::Backup {
    method say_hello {
        say("Hello!");
    }
}

my $backup_obj = MyUtil::Backup->new();
$backup_obj->say_hello;

We see this:

Hello!

Now let’s add a field to hold our name, and use it in our greeting:

use strict;
use warnings;

use feature 'class';
use feature 'say';

no warnings 'experimental::class';

class MyUtil::Backup {

    field $name : param;

    method say_hello {
        say("Hello, $name!");
    }
}

my $backup_obj = MyUtil::Backup->new( name => 'Jim' );
$backup_obj->say_hello;

We’ve added a field named $name. param indicates that the field value will be initialized in the constructor, which we call as MyUtil::Backup->new( name => 'Jim' ).

When we run, we see this:

Hello, Jim!

Now we’ve learned enough to refactor our structured script, and make it object-oriented:

use strict;
use warnings;

use feature 'class';
use feature 'say';

no warnings 'experimental::class';

class MyUtil::Backup {

    method exec_backup ( $source, $target ) {
        my $proc_name = "rsync -lrtv --delete \"${source}\" \"${target}\"";

        unless ( -d $target ) {
            mkdir($target);
        }

        if ( -d $target ) {
            say("Syncing ${source}...");

            system($proc_name);

            say("Synced ${source}");
        }

        say("----------");
    }

    method exec_backup_set ( $target_path, @source_paths ) {
        foreach my $source_path (@source_paths) {
            $self->exec_backup( $source_path, $target_path );
        }
    }
}

my $backup_obj = MyUtil::Backup->new();

my @target_paths =
  ( "/target1/", "/target2/" );
 
# regular file sets
my @regular_files = (
    "/home/jimc/source1",  "/home/jimc/source2",
    "/home/jimc/source3",  "/home/jimc/source4"
);

$backup_obj->exec_backup_set( $target_paths[0], @regular_files );
 
# large file sets
my @large_files = ( "/home/jimc/large1", "/home/jimc/large2" );

$backup_obj->exec_backup_set( $target_paths[1], @large_files );

sleep(2);  # pause before exiting

This one’s a bit more verbose, but still pretty clean and readable.

Note the syntax of the call to the exec_backup method from the exec_backup_set method: I had to prefix it with $self to indicate that a method of the current instance of the class is being called.

The Verdict

I still don’t consider Perl to be the friendliest language in the world, but taking advantage of modern features definitely improves the experience!

]]> Perl https://jcarr.codeberg.page/blog/posts/2024/modern_perl.html Mon, 01 Jul 2024 04:00:00 GMT 6502 Programming in C Jim Carr https://jcarr.codeberg.page/blog/posts/2024/6502_programming_c.html Would you like to learn about programming on a very old microprocessor?

In this Knowledge Base article, I’ll discuss programming for the 6502:

  1. Setting up a programming environment.
  2. Targeting the 6502 using the C language (and some Assembly)
  3. Testing using the 8-Bit Workshop IDE.
]]> Retro C Assembly https://jcarr.codeberg.page/blog/posts/2024/6502_programming_c.html Sun, 16 Jun 2024 04:00:00 GMT Think Async Jim Carr https://jcarr.codeberg.page/blog/posts/2024/think_async.html Usually, when writing asynchronous code, benefits are incremental. It can be difficult to see significant results at the micro level, unless you’re willing to dig in with profiling tools. Sometimes, though, benefits are obvious. I love these, because they provide real motivation for me to take the time to write better code.

A while back, I wrote a simple application to get unread mail counts for a list of accounts. I originally wrote it in Python, but I had some issues with it not always playing nice when scheduled in Conky. I rewrote it as a console application in C#, and got much better stability.

Originally, it was structured as follows. Information about an account is populated in a MailEntry class:

public class MailEntry
{
    public MailEntry(string hostUrl, string userName, string password)
    {
        HostUrl = hostUrl;
        UserName = userName;
        Password = password;
    }
 
    public string? HostUrl { get; set; }
    public string? UserName { get; set; }
    public string? Password { get; set; }
}

I store my account info in appsettings, and populate a list:

List<MailEntry> mailEntries = new();
 
// code to retrieve from appsettings goes here.

Then, I iterate through the list and use my MailHandler code to do the actual checking and reporting of unread counts:

foreach (MailEntry mailEntry in mailEntries)
{
    mailHandler.CheckMail(
        mailEntry.HostUrl ?? "",
        mailEntry.UserName ?? "",
        mailEntry.Password ?? ""
    );
}

But, you’ll notice that this code runs in a linear fashion. As in:

  1. Make a call to check an account.
  2. Wait for that call to complete.
  3. Move to the next item in the list.
  4. Repeat.

I have 10 accounts that I’m checking, so this is not efficient at all. It wasn’t a big deal, but it started to bother me that it was taking 10 - 15 seconds to complete on each run. I decided to do some refactoring to support asynchronicity. First, I changed Main():

private static async Task Main(string[] args)

Then, I created a list to hold information about the state of each running task:

List<Task> tasks = new();

I changed the CheckMail method in MailHandler:

public async Task CheckMail(string hostUrl, string userName, string password)

I checked each method I’m calling in the IMAP library I’m using to see which ones have an asynch version, and updated them accordingly:

await client.AuthenticateAsync(userName, password);
 
await client.Inbox.OpenAsync(FolderAccess.ReadOnly);

Back in Main(), the check loop is updated to add each async task to the list:

foreach (MailEntry mailEntry in mailEntries)
{
    tasks.Add(
        mailHandler.CheckMail(
            mailEntry.HostUrl ?? "",
            mailEntry.UserName ?? "",
            mailEntry.Password ?? ""
        )
    );
}

And then I wait for all the tasks to complete before proceeding:

await Task.WhenAll(tasks);

My application now runs in 2 - 4 seconds. The takeaway here is that it’s absolutely worth your time to look carefully at repeating logic in your code and optimize it to run asynchronously, if you can.

Full code is here: https://github.com/jfcarr/unread-mail-count

]]> .NET https://jcarr.codeberg.page/blog/posts/2024/think_async.html Sun, 09 Jun 2024 04:00:00 GMT DuckDuckGo AI Chat Jim Carr https://jcarr.codeberg.page/blog/posts/2024/ddg_ai_chat.html When you execute a search on DuckDuckGo, you’ll now see a new option at the top of the results:

Clicking “Chat” gives you a UI very similar to other AI Chatbots. (You can also send your query directly to the Chat UI by including a chat bang in your query text, e.g., What are the pros and cons of AI? !chat.)

When the Chat UI is displayed, you’ll notice a comforting bit of text on the left-hand side:

Active Privacy Protection Your chats are private, never saved by us, and not used to train AI models.

You can choose from two different chat models:

I seem to be getting slightly better results from Claude.

Caveats

Before you use an AI chatbot, consider the following:

  • Ethical Concerns In order to build its LLM, an AI engine scrapes massive amounts of information from the web. There are two problems with this:
    • The quality of data sources varies a great deal. And, even when the quality is good, you’re at the mercy of the AI engine’s ability to contextualize it properly. It is, after all, only a large statistical model.
    • The AI engine doesn’t provide credit to the original source. (This is particularly problematic in code generation, as you may be using code provided by the AI without respecting the original license of the source it was scraped from.)
  • Overcompensation In an effort to “do the right thing”, results can seriously misfire.
  • Curation The answers you get are filtered through the lens of the AI. Unlike a web search, where you receive results from many sources and can pick-and-choose which of those you want to follow through on, asking your questions via the AI means the AI preselects the answers for you.

If you still want to make use of AI Chat, but via a delivery that’s more privacy-centric, this seems to be a good choice.

]]> AI https://jcarr.codeberg.page/blog/posts/2024/ddg_ai_chat.html Mon, 15 Apr 2024 04:00:00 GMT Practical Astronomy Algorithms in Various Languages Jim Carr https://jcarr.codeberg.page/blog/posts/2024/practical_astro_languages.html For several years, I’ve worked off-and-on to implement astronomical algorithms from the Practical Astronomy book in various languages. I’ve just completed a JavaScript version, and I’ve published it to NPM. I’ve included a couple of simple code examples in the repo README, including how to calculate details for the April 8 solar eclipse.

Background

The algorithms are described in detail in the Practical Astronomy with your Calculator or Spreadsheet book, by Peter Duffett-Smith. I highly recommend that you get a copy of the book, as it provides lots of explanations and context that you won’t get by looking at the code alone. I worked with the 4th edition.

My code is actually a translation of macros from accompanying spreadsheet resources. You can download the spreadsheets from here. All Languages

These are all the languages for which I’ve implemented the Practical Astronomy algorithms, and the status of each function’s completion (as of June 1, 2024).

Function C C++ .NET/C# PHP Python Java JavaScript Rust
Date/Time
Calculate -> Date of Easter x x x x x x x x
Convert -> Civil Date to Day Number x x x x x x x x
Convert -> Civil Time <-> Decimal Hours x x x x x x x x
Extract -> Hour, Minutes, and Seconds parts of Decimal Hours x x x x x x x x
Convert -> Local Civil Time <-> Universal Time x x x x x x x x
Convert -> Universal Time <-> Greenwich Sidereal Time x x x x x x x x
Convert -> Greenwich Sidereal Time <-> Local Sidereal Time x x x x x x x x
Coordinates
Convert -> Angle <-> Decimal Degrees x x x x x x x x
Convert -> Right Ascension <-> Hour Angle x x x x x x x x
Convert -> Equatorial Coordinates <-> Horizon Coordinates x x x x x x x x
Calculate -> Obliquity of the Ecliptic x x x x x x x x
Convert -> Ecliptic Coordinates <-> Equatorial Coordinates x x x x x x x x
Convert -> Equatorial Coordinates <-> Galactic Coordinates x x x x x x x x
Calculate -> Angle between two objects x x x x x x x x
Calculate -> Rising and Setting times for an object x x x x x x x x
Calculate -> Precession (corrected coordinates between two epochs) x x x x x x x x
Calculate -> Nutation (in ecliptic longitude and obliquity) for a Greenwich date x x x x x x x x
Calculate -> Effects of aberration for ecliptic coordinates x x x x x x x x
Calculate -> RA and Declination values, corrected for atmospheric refraction x x x x x x x x
Calculate -> RA and Declination values, corrected for geocentric parallax x x x x x x x x
Calculate -> Heliographic coordinates x x x x x x x
Calculate -> Carrington rotation number x x x x x x x
Calculate -> Selenographic (lunar) coordinates (sub-Earth and sub-Solar) x x x x x x x
The Sun
Calculate -> Approximate and precise positions of the Sun x x x x x x x
Calculate -> Sun’s distance and angular size x x x x x x x
Calculate -> Local sunrise and sunset x x x x x x
Calculate -> Morning and evening twilight x x x x x x
Calculate -> Equation of time x x x x x x
Calculate -> Solar elongation x x x x x x
Planets
Calculate -> Approximate position of planet x x x x x x
Calculate -> Precise position of planet x x x x x
Calculate -> Visual aspects of planet (distance, angular diameter, phase, light time, position angle of bright limb, and apparent magnitude) x x x x x
Comets
Calculate -> Position of comet (elliptical) x x x x x
Calculate -> Position of comet (parabolic) x x x x x
Binary Stars
Calculate -> Binary star orbit data x x x x x
The Moon
Calculate -> Approximate and precise position of Moon x x x x x
Calculate -> Moon phase and position angle of bright limb x x x x x
Calculate -> Times of new Moon and full Moon x x x x x
Calculate -> Moon’s distance, angular diameter, and horizontal parallax x x x x x
Calculate -> Local moonrise and moonset x x x x x
Eclipses
Calculate -> Lunar eclipse occurrence and circumstances x x x x x
Calculate -> Solar eclipse occurrence and circumstances x x x x x

Repositories

]]> Astronomy https://jcarr.codeberg.page/blog/posts/2024/practical_astro_languages.html Wed, 03 Apr 2024 04:00:00 GMT Modern .NET Tooling in VS Code Jim Carr https://jcarr.codeberg.page/blog/posts/2023/modern_dotnet_tooling.html .NET / VS Code session incoming! I’ve covered this topic before, but now there are some interesting newer features in the “C# Dev Kit” extension. Come to the Innovation Hub in Dayton on November 14th, and learn all about it!

The Dayton .NET Developers group is part of GemCity TECH. There are groups covering a wide variety of technical topics and interests. Check it out!

]]> .NET Visual Studio Code https://jcarr.codeberg.page/blog/posts/2023/modern_dotnet_tooling.html Sun, 29 Oct 2023 04:00:00 GMT .NET IoT and Terminal GUI Jim Carr https://jcarr.codeberg.page/blog/posts/2023/dotnet_iot_terminal_gui.html I gave this talk for the Dayton .NET Developers Group at the Innovation Hub (in Dayton, Ohio) on August 8th. I’m really happy with how this one turned out! We were able to get through all of the content, and without technical glitches! Good conversation afterwards, too.

If you’re interested in the content, all of the source code is available at https://github.com/jfcarr/dotnet-iot-tui, along with high-level documentation at https://jfcarr.github.io/dotnet-iot-tui/.

Time to start thinking about what’s next!

]]> .NET Embedded and IoT https://jcarr.codeberg.page/blog/posts/2023/dotnet_iot_terminal_gui.html Fri, 11 Aug 2023 04:00:00 GMT