tag:blogger.com,1999:blog-4432237395942490292024-02-21T03:42:42.255-08:00Dissasemble #5Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-443223739594249029.post-48618968195798075282018-05-25T13:16:00.000-07:002018-05-25T13:16:41.931-07:00Interesting Ansible ipify problem<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;">I had this interesting problem recently and was able to work around that problem (once I discovered it)</span></span><br />
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></span>
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;">I was using the ansible functionality of 'ipify' to determine the public ip address of the host I was provisioning. It worked great and I used that fact to find the ethernet interface that was attached to that public ip address. </span></span><br />
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></span>
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;">I ran it on a new host and it <i><b>just didn't work</b></i>.</span></span><br />
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></span>
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;">After far too long of troubleshooting I discovered my problem. The ip address provided by 'ipify` was <i><b>not</b></i> the public ip address of the host I ran the playbook on. It was the public ip address of the http proxy that the system was using (...doh!!!). Don't get me wrong, this is the correct behavior of an application that uses http by respecting the proxy settings. However, ipify has no way to override the proxy setting.</span></span><br />
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;"><br /></span></span>
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;">I was able to solve it:</span></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span>
<span style="font-size: x-small;"><span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-family: "Courier New", Courier, monospace;">- name: delete public ip address file<br /> file:<br /> state: absent<br /> path: /tmp/public_ip_address<br /><br />- name: set_facts | get my public IP<br /> get_url:<br /> url: http://ifconfig.co/ip<br /> use_proxy: no<br /> dest: /tmp/public_ip_address<br /><br />- name: Slurp file with public ip address<br /> slurp:<br /> src: /tmp/public_ip_address<br /> register: slurpfile<br /> <br />- name: set_facts | set fact from ip address in slurped file<br /> set_fact:<br /> _public_ip_address: "{{ slurpfile['content'] | b64decode | ipaddr }}" <br /><br />- name: set_facts | interface for public ip<br /> set_fact:<br /> public_interface: "{{ item }}"<br /> when: ><br /> (hostvars[inventory_hostname]['ansible_%s' % item]|default({}))<br /> .get('ipv4', {}).get('address') == _public_ip_address <br /> or<br /> _public_ip_address in ((hostvars[inventory_hostname]['ansible_%s' % item]|default({}))<br /> .get('ipv4_secondaries'))|map(attribute='address')|list<br /> with_items:<br /> - "{{ ansible_interfaces }}"</span></span></span><br />
<br />
<br />
<br />
<span style="font-size: large;"><span style="font-family: Arial, Helvetica, sans-serif;">Many thanks to user larsks that published a good way to iterate through the interfaces.</span></span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><a href="https://stackoverflow.com/questions/45659201/discover-linux-interface-name-based-on-ip-address?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa" target="_blank">StackOverFlow discussion about Ansible iterating interface details</a></span><br />
<br />
<span style="font-family: Arial, Helvetica, sans-serif;"><span style="font-size: large;">And shoutout to Flowroute ( <a href="https://www.flowroute.com/" target="_blank">https://www.flowroute.com/</a> ) </span></span>Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-70282826140733285892018-01-29T10:31:00.000-08:002018-01-29T10:33:05.979-08:00Using Docker to solve my JAVA woes<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Occasionally you encounter a problem that resurfaces every 6 months or so that you wish never would come back again.
One of those problems is running multiple versions of Java (for various reasons) and toggling between those versions. "Jenv" works pretty well on a Mac, but sometimes when you are talking about libraries and java plugins it goes back to being complex.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">For this last round, I took a different approach. What if I created a docker container that had everything I needed for <i>that task</i>? Then I could reuse that container whenever I needed and did not need to make any changes to my host system. My challenge this time around was connecting to an Avocent KVM switch that could only support a very old version of JAVA webstart.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">I started with a basic Debian Wheezy container and added all the JAVA parts that I needed, then setup VNC to connect to the container. Downloading and launching the container is pretty easy if you follow the directions here https://hub.docker.com/r/paklids/jnlp-helper/ .</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">After that, there are a few other steps to follow.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">#1 VNC to the container. I'm using RealVNC on a Mac, but you may use another method (My coworker used the Safari browser)</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">#2 After the window opens (using the correct password) then launch firefox from the text console
</span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjucgLPkxH_ymlLhI_AIPsF5RVOBxuCjyhA_B3bcxKa_PRJidqaIUp2M1Q0p9AsOg1GvFRxjbevKXxStwO3qzoeAUM4ORMqe_KUJLF7ketU0Gx0aH25zKh75hDOexXdTFBw9nsLqrGNA7Q/s1600/Screen+Shot+2018-01-26+at+11.10.18+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="790" data-original-width="1029" height="307" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjucgLPkxH_ymlLhI_AIPsF5RVOBxuCjyhA_B3bcxKa_PRJidqaIUp2M1Q0p9AsOg1GvFRxjbevKXxStwO3qzoeAUM4ORMqe_KUJLF7ketU0Gx0aH25zKh75hDOexXdTFBw9nsLqrGNA7Q/s400/Screen+Shot+2018-01-26+at+11.10.18+AM.png" width="400" /></a></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">#3 Use firefox to browse to your Avocent KVM (or whatever site you need the older JAVA for - like those that use JNLP's)
</span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVmUYQ7vIIbcLaHdxsW3bbuJdDflxQQeFUk4YAyYno1B9XFPNIoUNyTqCOik_efgKS5Ucdi5q9IroSZr7lqp5hyphenhyphency6bPrXLt9xDJ-8ghe262Zb_Ax45Ih_RgW3ldcO-tKPRNpJarugmSc/s1600/Screen+Shot+2018-01-29+at+9.20.06+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="795" data-original-width="1028" height="309" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVmUYQ7vIIbcLaHdxsW3bbuJdDflxQQeFUk4YAyYno1B9XFPNIoUNyTqCOik_efgKS5Ucdi5q9IroSZr7lqp5hyphenhyphency6bPrXLt9xDJ-8ghe262Zb_Ax45Ih_RgW3ldcO-tKPRNpJarugmSc/s400/Screen+Shot+2018-01-29+at+9.20.06+AM.png" width="400" /></a></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">#4 Accept any SSL exceptions. Even if you resolve the SSL problems with your appliance, you will still need to use an older version of JAVA which in itself is likely insecure. Now login to your device:
</span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEv82bcpccfJSG-TFIOUUdaPMBRhEBBbpE1CqqVeFbC-d771deNnJYv7izhiJOMlUvyaoU-sX3CYa5oseZSWAWJBmYNIQopFiA4KzEX32SVz_Gsp545izXSeGUrDhTcVg-0LkICR7M0Bo/s1600/Screen+Shot+2018-01-29+at+9.20.43+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="795" data-original-width="1028" height="309" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhEv82bcpccfJSG-TFIOUUdaPMBRhEBBbpE1CqqVeFbC-d771deNnJYv7izhiJOMlUvyaoU-sX3CYa5oseZSWAWJBmYNIQopFiA4KzEX32SVz_Gsp545izXSeGUrDhTcVg-0LkICR7M0Bo/s400/Screen+Shot+2018-01-29+at+9.20.43+AM.png" width="400" /></a></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">#5 Click on any of the links that uses Java Webstart (like JNLP links). Use the "Open with" context and select browse:
</span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ9fhF47JvlR1giZgagQNEh8Dyw7CeNUkyT9_dSQvYwlJ7VRNT3y9ARE2uFu5ZWtgsS5aPeFF-x6pBYKWfa3C28h__Y9urAGD1fjAVJVW0G_9CxWAikm1SMfG0ovPh0bNMF_SpDQGPd6U/s1600/Screen+Shot+2018-01-29+at+9.21.35+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="260" data-original-width="356" height="292" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ9fhF47JvlR1giZgagQNEh8Dyw7CeNUkyT9_dSQvYwlJ7VRNT3y9ARE2uFu5ZWtgsS5aPeFF-x6pBYKWfa3C28h__Y9urAGD1fjAVJVW0G_9CxWAikm1SMfG0ovPh0bNMF_SpDQGPd6U/s400/Screen+Shot+2018-01-29+at+9.21.35+AM.png" width="400" /></a></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">#6 Show other applications and then select "IcedTea Java Web Start"
</span><br />
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-DDlCGDcDABLf3jcnrl1C0OI730PgnXhQw-lpClrJrZmCGqTN7Kx7aHVzaQBmuOxH1bf80HmAGd7k2NorWZ6F36CDs9e38fyNE6FC1wnN7ppgBXlbmcYOBzf9OYW6fdVI5i-GG9kLUuo/s1600/Screen+Shot+2018-01-29+at+9.21.52+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="449" data-original-width="430" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-DDlCGDcDABLf3jcnrl1C0OI730PgnXhQw-lpClrJrZmCGqTN7Kx7aHVzaQBmuOxH1bf80HmAGd7k2NorWZ6F36CDs9e38fyNE6FC1wnN7ppgBXlbmcYOBzf9OYW6fdVI5i-GG9kLUuo/s400/Screen+Shot+2018-01-29+at+9.21.52+AM.png" width="383" /></a></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif_LzA-DaJ9NUCe4iRWo-rBPoyzjbjZo5NCcygkcypoWQ4YVertJWvcnmv6C6tvfPXH6MXh6d86j1YRfPOJrqYkM4jFfCvlWdCbrLTKPGn4XqYe6C8K30oz9xZ3_8_MyrczIC0gW1oeRE/s1600/Screen+Shot+2018-01-29+at+9.22.03+AM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="449" data-original-width="426" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif_LzA-DaJ9NUCe4iRWo-rBPoyzjbjZo5NCcygkcypoWQ4YVertJWvcnmv6C6tvfPXH6MXh6d86j1YRfPOJrqYkM4jFfCvlWdCbrLTKPGn4XqYe6C8K30oz9xZ3_8_MyrczIC0gW1oeRE/s400/Screen+Shot+2018-01-29+at+9.22.03+AM.png" width="380" /></a></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">At this point your Java Webstart application should start. I'll try to publish my Dockerfile so that if you need to tweak for your own purposes then you have a good base to start from.
Enjoy!
</span>Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com3tag:blogger.com,1999:blog-443223739594249029.post-4761896287673361202018-01-13T18:50:00.001-08:002018-01-13T18:53:56.821-08:00My first iPXE adventure!<span style="font-family: "arial" , "helvetica" , sans-serif;">I recently encountered the need to rebuild some bare-metal servers in one of our datacenters and fell into a strange requirement. I've built tons of PXE boot systems before (all different variations of kickstart) so I was stubborn when the recommendation for iPXE came along. </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">"I can do this with standard PXE!" I said...</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">I was wrong.</span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">This time around I needed iPXE for passing arguments to a dynamic build system. Initially, I was resistant. Then I bit the bullet and jumped into using iPXE. Boy was I glad, because it gave me flexibility that I hadn't used before. I'll document the process I used to configure my build box, but I wanted first to post my working iPXE boot config (also its the boot menu)</span><br />
<br />
<br />
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); overflow: auto; padding: 5px; width: 653px;"><span style="font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace;"><span style="font-size: 12px; line-height: 14px;">
</span></span><code style="word-wrap: normal;"><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">
#!ipxe<br /># pulled from http://boot.ipxe.org/undionly.kpxe<br />set store mybuildserver.example.com<br /><br />prompt --key 0x02 --timeout 1000 Press Ctrl-B for the iPXE command line... && shell ||<br /><br />:boot_menu<br />menu iPXE Boot Menu<br />item localboot Boot From Local Disk<br />item --gap -- --------- Operating Systems -------------<br />item ubuntu1604 Wipe and Install Ubuntu 16.04.3<br />item coreos Wipe and Install CoreOS<br />item --gap -- --------- Utilities -------------<br />item gparted GParted Partition Manager<br />item DBAN Dariks Boot and Nuke NOTE: at end press Alt-F4 and reboot<br />item --gap -- --------- iPXE tools -------------<br />item shell iPXE Shell<br />item reboot Reload iPXE<br /><br />choose --default localboot --timeout 30000 target && goto ${target} ||<br />echo __NOTE: Cancel Enter Select Menu, Exit<br />exit<br /><br />:localboot<br />sanboot --no-describe --drive 0x80 || goto boot_menu<br /><br />:ubuntu1604<br />echo Starting Ubuntu Xenial installer for ${mac}<br />kernel http://${store}/ubuntu/install/netboot/ubuntu-installer/amd64/linux<br />initrd http://${store}/ubuntu/install/netboot/ubuntu-installer/amd64/initrd.gz<br />imgargs linux auto=true url=http://${store}/auto-16-preseed.cfg priority=critical preseed/interactive=false netcfg/choose_interface=enp3s0 vga=788 live-installer/net-image=http://${store}/ubuntu/install/filesystem.squashfs <br />boot || goto boot_menu<br /><br />:coreos<br />kernel http://${store}/coreos/coreos_production_pxe.vmlinuz<br />initrd http://${store}/coreos/coreos_production_pxe_image.cpio.gz <br />imgargs coreos_production_pxe.vmlinuz coreos.first_boot=1 coreos.autologin coreos.config.url=http://${store}:8080/ignition?mac=${mac:hexhyp}<br />boot || goto boot_menu<br /><br />:gparted<br />kernel http://${store}/gparted/live/vmlinuz<br />initrd http://${store}/gparted/live/initrd.img<br />imgargs vmlinuz boot=live config components union=overlay username=user noswap noeject ip= vga=788 fetch=http://${store}/gparted/live/filesystem.squashfs<br />boot || goto boot_menu<br /><br />:DBAN<br />kernel http://${store}/dban/dban.bzi<br />#imgargs dban.bzi nuke="dwipe"<br />imgargs dban.bzi nuke="dwipe --autonuke --method=zero" silent<br />boot || goto boot_menu<br /><br />:shell<br />echo __NOTE: Type 'config' enter iPXE config setting, 'exit' return to boot menu.<br />shell<br />goto boot_menu</span></span></pre>Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-54670392116058599262017-10-03T11:43:00.000-07:002017-10-03T12:36:38.551-07:00A reliable public API for (close to) free!<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Sorry for the lack of posts lately, but I've been busy churning out projects (both personal and business related).</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">I recently went through an exercise that will definitely benefit some of the communities that I interact with (HAD and CoffeeOps) and felt obligated to share because I'm super excited about it.</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">In the world of IOT and scalable applications, I don't need to spend much time hyping the benefits of a simple API. Along those lines I've been playing around with the Serverless Framework to spin up services (in my case AWS Lambda) but then asked myself 3 distinct questions:</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">1. How small is "too small" for an API?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">2. How cheap could I make a small API?</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">3. How reliable (or performant) could I make an API?</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">This seems very similar to the argument about hiring a contractor ("You can have cheap, good or fast....but you can only pick 2 of the 3 options), but it turns out that when you go small you can get pretty close to all three.</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">With Serverless Framework, in AWS it is pretty easy to setup an API Gateway that passes input to a Lambda function. It's really small and granular and AWS can host it on their infrastructure really easily. That covers the small stuff.</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">And here is the plus for us -<i> it's really cheap</i>! You pay only for the number of hits to your API Gateway and most AWS accounts give you a minimum number of Lambda functions for free (usually in the thousands or millions). But my problem was the last part - where do you store your data? You could spin up an RDS like PostgreSQL or MySQL, a Redis database or even a DynamoDB. But can you go cheaper than that? Yes....yes you can. </span><br />
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">AWS offers a simple key value store that is most often used for configuration data called SSM Parameter store. In that KV store, you can store strings of up to 4096 characters (I don't think it is bytes according to what I've found online) which is more than enough for the purpose I have. Using parameter store should be a free built in service, so that covers the whole cheap argument.</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">So finally we get to the reliable and fast part. Depending on how you write your function, you can have an API that completes well under 100ms. As for reliability, AWS has a long history of reliable services, and that isn't even counting that this could run in multiple regions (as long as you could have a way to sync your data). Reliable....check.</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">So how do you do it? Here is my very basic working example (no authentication in this example BTW...only a proof of concept):</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Step 1:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Learn about and install Serverless Framework here https://serverless.com/ . Assumed is that you already have an AWS account to work with, but if not, you'll need to set one up.</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Step 2:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">You can create a serverless project using the example templates, but to start out with you only need 3 things....a directory to work in, a serverless.yml file and a handler (I'm using python for mine)</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Step 3:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Create a Parameter in the SSM Parameter Store. Its in the AWS console under the EC2 section. Not hard to find and you only need to set an un-encrypted string. I'm using the name 'myParam'. </span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Step 4: A bit of code (edit your account & region as needed):</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-size: large;"><b><span style="font-family: "arial" , "helvetica" , sans-serif;">serverless.yml </span></b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">
</span><br />
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); color: black; font-family: "andale mono","lucida console","monaco","fixed",monospace; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"> <code style="color: black; overflow-wrap: normal;">
# Welcome to Serverless!
#
service: param-api # NOTE: update this with your service name
# frameworkVersion: "=X.X.X"
provider:
name: aws
runtime: python2.7
stage: dev
region: us-west-2
iamRoleStatements:
- Effect: "Allow"
Action:
- "ssm:GetParameters"
- "ssm:PutParameter"
Resource:
- "arn:aws:ssm:${self:provider.region}:${accountId}:parameter/myParam"
functions:
get_myParam:
handler: handler.get_myParam
events:
- http:
path: get
method: get
put_myParam:
handler: handler.put_myParam
events:
- http:
path: put
method: put
</code>
</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">
</span>
<br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">and my python functions in <b>handler.py</b></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">
</span>
<br />
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); color: black; font-family: "andale mono","lucida console","monaco","fixed",monospace; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><span style="font-family: "arial" , "helvetica" , sans-serif;"> <code style="color: black; overflow-wrap: normal;">
import json
import boto3
def get_myParam(event, context):
ssmResponse = boto3.client('ssm').get_parameters(
Names=['myParam'],
WithDecryption=False
)
body = {
"message": ssmResponse['Parameters'],
"input": event['body']
}
response = {
"statusCode": 200,
"body": json.dumps(body)
}
return response
def put_myParam(event, context):
ssmResponse = boto3.client('ssm').put_parameter(
Name='myParam',
Description='this is a test',
Value=event['body'],
Type='String',
Overwrite=True
)
response = {
"statusCode": 200,
"body": json.dumps(ssmResponse)
}
return response</code></span>
</span></pre>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"> <b>Step 5:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Deploy using serverless. At CLI "serverless deploy -v"</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Step 6:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Test. Curl against the URL that you were given after your deploy completed. This API now supports get and put, but you could add whatever you want as far as HTTP method and authentication.</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><b>Step 7:</b> </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Profit!</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">What could you do with this? Any number of things. Need to share data across IOT devices? Check. Need to build a global service and you don't have datacenters across the globe? Check. The advantage with this is that the investment is very low and this scales to extremely high (just on the micro level, like owning 50 million insects that you control all over the world).</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Many thanks to my employer (<a href="https://www.flowroute.com/">https://www.flowroute.com/</a> ) for letting me work on this and my coworker Reed for helping me through the Python bits and bobs. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: large;">Also, thanks to the Seattle CoffeeOps meetup <a href="https://www.meetup.com/Seattle-CoffeeOps/">https://www.meetup.com/Seattle-CoffeeOps/</a> for listening to me rant about this stuff even though they are asking "What is he talking about ?!?"</span><br />
<span style="font-size: large;"><br /></span>
<span style="font-size: large;"><br /></span>Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com1tag:blogger.com,1999:blog-443223739594249029.post-69336232934498941012016-03-02T14:36:00.000-08:002016-03-02T14:40:40.177-08:00Setting up an apt-mirror for Cumulus Linux<span style="font-family: "arial" , "helvetica" , sans-serif;">You may want to setup a local apt repository to pull packages from for your Cumulus infrastructure. </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I followed the instructions found here: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">https://www.howtoforge.com/local_debian_ubuntu_mirror </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">But I did find a few gotcha's that I had to fix before I was able to use my local repo. Here is a sample of my /etc/apt/mirror.list</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<br />
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); overflow: auto; padding: 5px; width: 653px;"><span style="font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace;"><span style="font-size: 12px; line-height: 14px;">
</span></span><code style="word-wrap: normal;"><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">############# config ##################
#
# set base_path /var/spool/apt-mirror
#
# set mirror_path $base_path/mirror
# set skel_path $base_path/skel
# set var_path $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch <running architecture="" host="">
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads 20
set _tilde 0
#
############# end config ##############
deb http://repo.cumulusnetworks.com CumulusLinux-2.5 main addons updates
deb http://repo.cumulusnetworks.com CumulusLinux-2.5 security-updates
# Uncomment the next line to get access to the testing component
# deb http://repo.cumulusnetworks.com CumulusLinux-2.5 testing
# Uncomment the next line to get access to the Cumulus community repository
# deb http://repo.cumulusnetworks.com/community/ CumulusLinux-Community-2.5 main addons updates
deb http://ftp.us.debian.org/debian wheezy-backports main
deb http://ftp.us.debian.org/debian/ wheezy main
# mirror additional architectures
#deb-alpha http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-amd64 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-armel http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-hppa http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-i386 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-ia64 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-m68k http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-mips http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-mipsel http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-powerpc http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-s390 http://ftp.us.debian.org/debian unstable main contrib non-free
#deb-sparc http://ftp.us.debian.org/debian unstable main contrib non-free
clean http://ftp.us.debian.org/debian</running></span></span></code><span style="font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace;"><span style="font-size: 12px; line-height: 14px;">
</span></span></pre>
</div>
<div>
<br /></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">I was able to use the instructions in the howto to build the mirror and present it via http. I did have some problems using it at first. Notice the error:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">W: Failed to fetch http://mydebianrepo.mycompany.com/debian/dists/wheezy-backports/main/i18n/Translation-en Hash Sum mismatch</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">And then I read this:</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">http://askubuntu.com/questions/217502/speed-up-apt-get-update-by-removing-known-ignored-translation-en</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">So to fix this put the line: </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Acquire::Languages "none"; </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif;">on the Cumulus node at the end of the conf file </span><span style="font-family: "arial" , "helvetica" , sans-serif;">/etc/apt/apt.conf.d/70debconf</span></div>
<div>
<br /></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">Here is my /etc/apt/sources.list</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span></div>
</div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<br />
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); overflow: auto; padding: 5px; width: 653px;"><span style="font-family: "andale mono" , "lucida console" , "monaco" , "fixed" , monospace;"><span style="font-size: 12px; line-height: 14px;"><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">#
#
deb http://mydebianrepo.mycompany.com/CumulusLinux CumulusLinux-2.5 main addons updates
deb http://</span></span><span style="color: #444444; font-size: 12px; line-height: 14px;">mydebianrepo.mycompany.com</span><span style="color: #444444; font-size: 12px; line-height: 14px;">/CumulusLinux CumulusLinux-2.5 security-updates</span><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">
</span></span><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">
# Uncomment the next line to get access to the testing component
# deb http://repo.cumulusnetworks.com CumulusLinux-2.5 testing
# Uncomment the next line to get access to the Cumulus community repository
# deb http://repo.cumulusnetworks.com/community/ CumulusLinux-Community-2.5 main addons updates
deb http://</span></span><span style="color: #444444; font-size: 12px; line-height: 14px;">mydebianrepo.mycompany.com</span><span style="color: #444444; font-size: 12px; line-height: 14px;">/debian wheezy-backports main</span><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">
</span></span><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">deb http://</span></span><span style="color: #444444; font-size: 12px; line-height: 14px;">mydebianrepo.mycompany.com</span><span style="color: #444444; font-size: 12px; line-height: 14px;">/debian wheezy main</span><span style="color: #444444;"><span style="font-size: 12px; line-height: 14px;">
</span></span><span style="font-size: 12px; line-height: 14px;">
</span></span></span></pre>
</div>
<div>
<br /></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<div>
</div>
</div>
</div>
Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-80533687790593326322016-03-01T10:51:00.000-08:002016-03-01T10:57:30.814-08:00Puppet 3 module for Cumulus Linux License<span style="font-family: "arial" , "helvetica" , sans-serif;">Here is a Puppet 3 module for applying the Cumulus Linux license to your nodes. You can change the logic to use $hostname or $fqdn or any other facter fact, but you will want to put some logic in there for future growth (ie: will you ever move to 40Gb switches?).</span><br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">License file is dropped on the node at /etc/license . Be sure to name your directory the SAME AS the name of the class in ../modulename/manifests/init.pp </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span>
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); font-family: 'andale mono', 'lucida console', monaco, fixed, monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 653px;">
<code style="word-wrap: normal;"><span style="color: #444444;"># manifests/init.pp
class cumulus_license {
## BASIC CHECK - ARE 10GB SWITCHES THIS SETUP? THEN USE 10GB LICENSE
## OTHERWISE USE OTHER FACTER FACT TO DECIDE
if ($architecture == 'x86_64') and ($operatingsystem == 'CumulusLinux') {
file { '/etc/license':
ensure => present,
owner => root,
group => root,
source => 'puppet:///modules/cumulus_license/my_10Gb_license',
}
## AND USE THIS IDENTIFIER TO DECIDE ON 1GB LICENSE
if ($architecture == 'ppc') and ($operatingsystem == 'CumulusLinux') {
file { '/etc/license':
ensure => present,
owner => root,
group => root,
source => 'puppet:///modules/cumulus_license/my_1Gb_license',
}
}
}
## CHANGE TO THE LICENSE FILE NOTIFIES THIS EXEC
## NOW RUN THE LICENSE COMMAND AND IF IT SUCCEEDS THEN DO NOTHING
## IF FAILS THEN HARDWARE IS UNLICENSED AND WILL INSTALL FROM PUPPET
exec { cl-license-command :
command => ['/usr/cumulus/bin/cl-license -i /etc/license'],
unless => ['/usr/cumulus/bin/cl-license'],
subscribe => File['/etc/license'],
refreshonly => true,
}
}
</span></code>
</pre>
</div>
<div>
<br /></div>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"></span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">A few notes about operation:</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<br />
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;"><br /></span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
</span>
<div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">1. First the puppet run validates that there is an existing valid license on the switch. If there is, then do nothing. If you want to apply a new license then you will update puppet and run the cli-license command manually</span></div>
<span style="font-family: "arial" , "helvetica" , sans-serif;">
<div>
2. If there is no active license then it tries to apply the license found at /etc/license (pushed there by puppet). You can rename this to whatever you need.<br />
3. Make sure your Cumulus License files are in the files section of the module. These are just simple text files that get pulled in by the cli_license command during the run<br />
4. You /may/ have to reboot the switch after the new license is applied. Please refer to Cumulus docs for guidance there.</div>
<div>
<br /></div>
<div>
<br /></div>
</span></div>
</div>
Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-5654034790182416772015-12-08T15:43:00.001-08:002015-12-10T09:56:22.491-08:00Cumulus VX on Libvirt (CentOS 6)<span style="font-family: "verdana" , sans-serif;">You may have seen my earlier post on Cumulus VX in libvirt. I'm going to go a step further and try to emulate the environment described here:</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">https://docs.cumulusnetworks.com/display/VX/Using+Cumulus+VX+with+KVM</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">There are some pretty significant differences though, because libvirt on CentOS 6 is pretty "feature light". Nevertheless, this should result in something to work with.</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">You will need to build a system that can serve DHCP and HTTP on your 'default' network and then disable the dhcp section (you could delete it) in /etc/libvirt/qemu/networks/default.xml . This is because while libvirt can create manual DHCP entries (sometimes called "static dhcp" entries) using the 'host' declaration, you will not be able to pass DHCP option 239 which is required for your Zero Touch Provisioning. Queue sad trombone noise.</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">Next, you will want to build 4 additional networks in the Virtual Machine Manager ( localhost>details>Virtual Networks ). I built isolated networks named testnet1 thru testnet4, but you don't need to follow my silliness. Be sure to restart libvirtd after you've completed this to be sure that it has picked up all your changes. Maybe backup your XML files too, because there are situations where libvirt can blow them away. Once your DHCP server is running on the 'default'<span style="font-family: "verdana" , sans-serif;"> network then build 4 manual DHCP entries using th<span style="font-family: "verdana" , sans-serif;">e MAC addresses that you'll find below for eth0 (the one<span style="font-family: "verdana" , sans-serif;">s that are on virbr0)</span>. </span></span></span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">You will follow the instructions in the cumulus docs but instead of the KVM commands you can use virt-install to initiate the guests (adjust path to qcow2 files you copied):</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">sudo virt-install --os-variant=generic --ram=256 --vcpus=1 --network bridge=virbr0,model=virtio,mac=00:01:00:00:01:01 --network network=testnet1,model=virtio,mac=00:00:02:00:00:11 --network network=testnet2,model=virtio,mac=00:00:02:00:00:12 --boot hd --disk path=/home/user1/leaf1.qcow2,format=qcow2 --name=leaf1</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">sudo virt-install --os-variant=generic --ram=256 --vcpus=1 --network bridge=virbr0,model=virtio,mac=00:01:00:00:01:02 --network network=testnet3,model=virtio,mac=00:00:02:00:00:21 --network network=testnet4,model=virtio,mac=00:00:02:00:00:22 --boot hd --disk path=/home/user1/leaf2.qcow2,format=qcow2 --name=leaf2</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">sudo virt-install --os-variant=generic --ram=256 --vcpus=1 --network bridge=virbr0,model=virtio,mac=00:01:00:00:01:03 --network network=testnet1,model=virtio,mac=00:00:02:00:00:31 --network network=testnet3,model=virtio,mac=00:00:02:00:00:32 --boot hd --disk path=/home/user1/spine1.qcow2,format=qcow2 --name=spine1</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">sudo virt-install --os-variant=generic --ram=256 --vcpus=1 --network bridge=virbr0,model=virtio,mac=00:01:00:00:01:04 --network network=testnet2,model=virtio,mac=00:00:02:00:00:41 --network network=testnet4,model=virtio,mac=00:00:02:00:00:42 --boot hd --disk path=/home/user1/spine2.qcow2,format=qcow2 --name=spine2</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;"> I will admit that the configs that are described in the Cumulus docs indicate that swp1 thru swp3 would be provisioned, but the virtual hardware configured is only swp1 thru swp2. I'm going to ignore swp3 for now.</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">If you built option 239 correctly on the DHCP scope, then you should be able to point all these instances to your ZTP script. Just remember that you do not need to run your web server on the deafult port 80, as ZTP will accept a path like 'http://192.168.122.101:1234/my_ztp_script.sh'</span><br />
<br />
<span style="font-family: "verdana" , sans-serif;">I haven't setup a configuration management tool on my instances yet, but that will be upcoming via my ZTP script. </span><br />
<span style="font-family: "verdana" , sans-serif;"><br /></span>
<span style="font-family: "verdana" , sans-serif;"><br /></span>
Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-77311271794524382402015-12-04T15:34:00.000-08:002015-12-10T09:56:40.633-08:00Boot Cumulus VX in a libvirt KVMWhy would I want to do that?<br />
<br />
Well, if you are running CentOS 6 and want an easy way to play with the Cumulus VX, here is a good start.<br />
<br />
On a CentOS 6 system running a GUI<br />
#yum groupinstall "Virtualization*"<br />
<br />
(this will install all your libvirt bits and pieces)<br />
<br />
Then download the Cumulus KVM image (it is a qcow2 image) and remember the path<br />
<br />
#sudo virt-install --os-variant=generic --ram=256 --vcpus=1 --network bridge=virbr0,model=virtio,mac=00:01:00:00:01:00 --boot hd --disk path=/home/user/cumulus.qcow2 --name=cumulus1<br />
<br />
This only provisions the eth0 interface (mgmt) but it should get you a start to playing with the product.<br />
<br />
Enjoy!!!Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-76531633969799924602015-09-15T13:02:00.001-07:002015-09-15T13:03:01.259-07:00Backup pfsense firewall (via SSH) using ONE script I know there are other methods out there to backup a pfsense config. These do work but I'm just not a fan of relying on the gui to perform my config backups. Plus, SSH is my encrypted session of choice because its secure, flexible and available.<br />
<br />
Once again I'm using Fabric to perform this backup. If you haven't already done it, go to the effort of setting up Fabric on your platform of choice...you won't regret it. If you want to do interesting things with Fabric then get warmed up on your python at Codecadamy (https://www.codecademy.com/tracks/python).<br />
<br />
This script uses an interactive prompt for you to enter the password, but you can simply provide the password either in the script itself (shame shame) or via another secure method.<br />
<br />
Backups are pulled back to the system running the script into a directory named 'my_pfsense_backups' and are given a directory for each day. You can tweak this to suit your needs.<br />
<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;">
<code style="color: black; word-wrap: normal;">
#!/usr/bin/python
#
# Designed and tested on pfsense v2.2
#
import urllib2, base64, getpass, json, re, sys, os
from fabric.api import *
from datetime import datetime
#
myname = ('root')
# NOTE: pfsense uses root user that has same password as admin - required for sftp file access
theList = ['pfsense1.company.com','pfsense2.company.com']
#
i = datetime.now()
now_is = i.strftime('%Y%m%d-%H%M%S')
today_is = i.strftime('%Y%m%d')
print now_is
#
print ('')
print ('Username is ' + myname)
pw = getpass.getpass()
print ('')
#
how_many = len(theList)
#
print("This will backup " + str(how_many) + " systems:\n")
print (theList)
print ('')
#
env.user = myname
env.hosts = theList
env.password = pw
#
#@parallel(pool_size=5)
#
# generate the backup file on the pfsense system itself, this will take some time
def generate_and_pull_backup():
env.warn_only = True
# run( "8", shell=False )
backup_command_output = run( "/etc/rc.create_full_backup", shell=False )
# parse the output of the create_full_backup command
file_generated_full_path = backup_command_output.rsplit(None, 1)[-1]
filename_generated = file_generated_full_path.split('/')[-1]
# pull the backup home to me
get("%s" % file_generated_full_path,"./my_pfsense_backups/%s/%s-%s" % (today_is,env.host,filename_generated))
# NOTE: configs can be restored via /etc/rc.restore_full_backup
#
# delete config backup just generated so disk does not fill
run( "rm -f %s" % file_generated_full_path, shell=False )
#
if __name__ == '__main__':
execute(generate_and_pull_backup)
</code>
</pre>
<br />
Hope you enjoy this as much as I have! Backing up my pfsense systems has always been far too manual and problem prone so I'm looking forward to putting that behind me.Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-23534751496856106602015-09-14T15:54:00.000-07:002015-09-14T15:56:44.156-07:00Backup F5 units (LTM or GTM) using one script!So in case you have gone down this rabbit hole, the BigIP series are great but certain things about them require finesse. Case in point - the management interface does not always play well with other published services like TACACS or NTP. Sometime they require custom routes and that can become a hassle.<br />
<br />
So backing up the config on an F5 can sometimes encounter these challenges too. I want to:<br />
<br />
1. Generate a config backup on my F5 unit using tmsh commands<br />
2. Securely transfer that config backup somewhere else, preferably on a secure network<br />
3. I don't want to deploy a webserver or entire development suite just to accomplish goals 1 or 2<br />
<br />
Enter Python Fabric.<br />
<br />
Yes, getting Fabric will take some setup....but this is an environment that you will be able to use over & over again (not just for F5 backups)<br />
<br />
The script you can grab below permits you to use Fabric to open an SSH session, generate the config backup and then pull it back (via SFTP) to the system where you originally ran the fabric script.<br />
<br />
Be sure to edit the user ("myname" variable) and the list of F5 units ("theList" variables) that you want to run this on. This can be used to be fully automated and headless but I'm going to let you work out those details yourself. Files will be stored on the local box in ./my_F5_backups under a date directory for each day or you could customize this script to go hand in glove with logrotate.<br />
<br />
<pre style="background-color: #eeeeee; border: 1px dashed #999999; color: black; font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace; font-size: 12px; line-height: 14px; overflow: auto; padding: 5px; width: 100%;"> <code style="color: black; word-wrap: normal;">
#!/usr/bin/python
#
# Designed and tested on LTM v11.x
#
import urllib2, base64, getpass, json, re, sys, os
from fabric.api import *
from datetime import datetime
#
i = datetime.now()
now_is = i.strftime('%Y%m%d-%H%M%S')
today_is = i.strftime('%Y%m%d')
print now_is
#
#
#
myname = ('my_admin_user')
print ('')
print ('Username is ' + myname)
pw = getpass.getpass()
print ('')
#
#
theList = ['ltm01.company.com','gtm01.company.com']
#
#
#
how_many = len(theList)
#
# I am using someone else's yes/no logic here
#
def query_yes_no(question, default="yes"):
valid = {"yes":"yes", "y":"yes", "ye":"yes",
"no":"no", "n":"no"}
if default == None:
prompt = " [y/n] "
elif default == "yes":
prompt = " [Y/n] "
elif default == "no":
prompt = " [y/N] "
else:
raise ValueError("invalid default answer: '%s'" % default)
while 1:
sys.stdout.write(question + prompt)
choice = raw_input().lower()
if default is not None and choice == '':
return default
elif choice in valid.keys():
return valid[choice]
else:
sys.stdout.write("Please respond with 'yes' or 'no' "\
"(or 'y' or 'n').\n")
#
keep_going = query_yes_no("This will backup " + str(how_many) + " systems. Would you like to continue?\n (No will list systems that it would have run on and exit)")
if keep_going == "no":
print (theList)
sys.exit()
print ('')
#
env.user = myname
env.hosts = theList
env.password = pw
#
#@parallel(pool_size=5)
#
# generate the backup file (ucs) on the F5 unit itself - will create some load on F5 while running
def run_on_f5_first():
run( "tmsh save sys ucs %s" % now_is, shell=True )
# copy backup file (ucs) back to local system
def file_get():
get("/var/local/ucs/%s.ucs" % now_is,"./my_F5_backups/%s/%s-%s.ucs" % (today_is,now_is,env.host))
# delete config backup just genrated so disk does not fill
def run_on_f5_last():
run( "rm -f /var/local/ucs/%s.ucs" % now_is, shell=True )
#
if __name__ == '__main__':
execute(run_on_f5_first)
execute (file_get)
execute(run_on_f5_last)
</code>
</pre>
<br />
And there you go. One script that does something that I've meant to do for years and just didn't get around to until now. Boy am I glad that I waited until Fabric came along!Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-57882942961816634302015-09-14T15:15:00.000-07:002015-09-14T15:17:59.183-07:00Python Fabric read hostnames from ForemanI don't think that it comes as any surprise that I love Fabric. It just <i><b>works</b></i>. Yeah, you have to know a bit of python to make things work the way you want it, but this isn't rocket science. If you have problems running a python script or a fabfile then you will find a large set of examples online.<br />
<div>
<br /></div>
<div>
One thing that you may find challenging in Fabric (at least I have) is keeping your host lists up to date. This is now not a problem is you leverage Foreman to store your system information!</div>
<div>
<br /></div>
<div>
This is an interactive script that pulls your system hostnames out of Foreman (using the rest API) and then allows you to run a regex against the hostnames to narrow that list down. I've used someone else's yes/no logic so I can't take credit for that one (sorry, I can't recall where I pulled it from - I wrote this a few weeks ago)</div>
<div>
<br /></div>
<div>
I've named this <b>sprinkler.py</b> but you can name it what you want. Be sure to edit your Foreman FQDN and mark the file as executable....and remember that you can do <i><b>great</b> &</i> <b>t<i>errible</i></b> things with this script:</div>
<div>
<br /></div>
<div>
<pre style="background-color: #eeeeee; border: 1px dashed rgb(153, 153, 153); overflow: auto; padding: 5px; width: 100%;"><span style="font-family: Andale Mono, Lucida Console, Monaco, fixed, monospace;"><span style="font-size: 12px; line-height: 12px;">
<code style="color: black; word-wrap: normal;">
#!/usr/bin/python
import urllib2, base64, sys, getpass, json, os, re
from fabric.api import *
from fabric.tasks import execute
##
#Set variables we need to use
##
string_o_machines = ""
if len(sys.argv) == 1:
print ("\n Usage ./sprinkler.py <username> is required\n")
sys.exit(0)
else:
myname = sys.argv[1]
#
foreman_fqdn = "my_foreman_hostname.company.com"
#
def query_yes_no(question, default="yes"):
valid = {"yes":"yes", "y":"yes", "ye":"yes",
"no":"no", "n":"no"}
if default == None:
prompt = " [y/n] "
elif default == "yes":
prompt = " [Y/n] "
elif default == "no":
prompt = " [y/N] "
else:
raise ValueError("invalid default answer: '%s'" % default)
while 1:
sys.stdout.write(question + prompt)
choice = raw_input().lower()
if default is not None and choice == '':
return default
elif choice in valid.keys():
return valid[choice]
else:
sys.stdout.write("Please respond with 'yes' or 'no' "\
"(or 'y' or 'n').\n")
#
print ("")
print ("Username is " + myname)
pw = getpass.getpass() #Get password from person running script
print ("")
#
print ("What command should I issue?")
send_command = raw_input('')
print ("")
#
print ("What systems should I send this to? Use regex like .*domain.com app0?.*.com")
domain = raw_input('')
print ("")
#
needs_sudo = query_yes_no("Does this command require sudo rights?")
#
request = urllib2.Request("https://" + foreman_fqdn + "/api/hosts?per_page=10000")
#
#
#
# You need the replace to handle encodestring adding a trailing newline
# (https://docs.python.org/2/library/base64.html#base64.encodestring)
base64string = base64.encodestring('%s:%s' % (myname, pw)).replace('\n', '')
request.add_header("Authorization", "Basic %s" % base64string)
json_obj = urllib2.urlopen(request)
#
#
#
data = json.load(json_obj)
hostlist = []
for item in data:
hostname = item['host']['name']
hostlist.append(hostname)
x=re.compile(domain)
sub_list = filter(x.match, hostlist)
#print sub_list
this_many = len(sub_list)
#
#
print ('')
keep_going = query_yes_no("Command will be send to " + str(this_many) + " systems. Would you like to continue?\n (No will list systems that it would have run on and exit)")
if keep_going == "no":
print (sub_list)
sys.exit()
#
#
env.user = myname
env.hosts = sub_list
env.password = pw
#
@parallel(pool_size=5)
def command_to_run():
if needs_sudo == "no":
with settings(warn_only=True):
run( send_command )
else:
with settings(warn_only=True):
sudo( send_command )
#
if __name__ == '__main__':
execute(command_to_run)</username></code></span></span><span style="color: black; font-family: 'Andale Mono', 'Lucida Console', Monaco, fixed, monospace; font-size: 12px; line-height: 12px;">
</span></pre>
</div>
<div>
<br /></div>
<div>
<br /></div>
<blockquote class="tr_bq">
Have fun using this! (You may also customize how many parallel systems this runs on at any given time by adjusting the pool size on that.)</blockquote>
Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-55778060870681687622013-12-17T15:31:00.003-08:002013-12-17T15:31:54.183-08:00Setup Trigger on CentOs\RHELSetting up trigger (<a href="https://github.com/trigger/trigger">https://github.com/trigger/trigger</a>) is no small task on a RHEL\CentOs system.<br />
<br />
Here is a basic setup bash script which you can use or modify to get up and running. I haven't tested it yet on a completely new system, but it should work without too much tweaking.<br />
<br />
<br />
#!/bin/bash<br />
## run this as root on a CentOs/RHEL system (tested on Centos 6.4 with rpmforge repos already setup)<br />
## grab epel repository and make it available in yum<br />
wget http://mirrors.solfo.com/epel/6/i386/epel-release-6-8.noarch.rpm<br />
rpm -i epel-release-6-8.noarch.rpm<br />
## update yum cache so it knows to use epel rpms<br />
yum update<br />
## install everything you need to install trigger<br />
yum -y install libxml2-devel libxml++-devel python-devel python-setuptools python-simpleparse python-twisted python-pip redis automake gcc<br />
## start redis in case you need it<br />
chkconfig redis on<br />
service redis start<br />
## use pip to install trigger<br />
pip install trigger<br />
## use pip to install twister (may be done already...but JIC)<br />
pip install twister<br />
## build config directory and populate with something (pip copies trigger files into /tmp/pip-build-root/trigger/)<br />
mkdir -p /etc/trigger<br />
cp /tmp/pip-build-root/trigger/conf/netdevices.xml /etc/trigger/netdevices.xml<br />
cp /tmp/pip-build-root/trigger/conf/bounce.py /etc/trigger/bounce.py<br />
cp /tmp/pip-build-root/trigger/conf/trigger_settings.py /etc/trigger/settings.py<br />
cp /tmp/pip-build-root/trigger/conf/autoacl.py /etc/trigger/autoacl.py<br />
cp /tmp/pip-build-root/trigger/tests/data/tackf /etc/trigger/.tackf<br />
##create a few dirs that you may need<br />
mkdir -p /data/firewalls<br />
mkdir -p /data/tftproot<br />
## now you need to edit your /etc/trigger/settings.py and set your DB type and details<br />
## run init_task_db to populate your DB with the basic tables that you'll need<br />
<br />
Don't forget, this is for my benefit as well. Now I'll have this post to look back on to see how I did this before.<br />
<br />
<br />
<br />
<br />
<br />Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com0tag:blogger.com,1999:blog-443223739594249029.post-41664688236469579042012-01-25T11:35:00.000-08:002012-01-25T12:49:59.307-08:00Securely backup a Cisco Firewall (ASA & FWSM)I have a problem with Cisco. Let me put it simply in this list:<div><br /></div><div>1. Cisco does not support SSH keys (at least not very well) across its fleet of firewalls and security devices. </div><div>2. Config backups: TFTP is convenient but NOT secure. FTP is convenient but NOT secure. HTTP can be convenient but still NOT secure. HTTPS can be secure but can be unwieldy to setup and support. SCP is pretty convenient on a *nix machine, and is definitely secure, but there is a catch. </div><div>3. SCP on Cisco security devices: While these security devices usually do support SCP, they cannot SCP their configs to a destination (from the security device itself). Also, you cannot pull the running-config directly via SCP.</div><div><br /></div><div>Addressing point #1 - using a password in an expect script is not extremely secure, but it beats the alternatives in almost every case. There are ways to secure strings (like passwords) in expect but you can decide for yourself whether these are needed.</div><div>Addressing point #3 - SCP can be used in the following way: Using expect, SSH to the device and save the running-config to a predefined location (root of the disk) and logout. Open a second shell and SCP the saved config back (from predefined location on device to predefined location on expect host).</div><div><br /></div><div>My script isn't the shining example of a secure backup, but I'm unimpressed with the alternatives to this.</div><div><br /></div><div>Here is an example of my firewall_backup.exp:</div><div><br /></div><div><div></div><div>#!/usr/bin/expect</div><div>#######################################################</div><div># Written by paklids 2012 01 12 #</div><div># Expect script to securely backup Cisco ASAs & FWSMs #</div><div># using SSH and SCP. #</div><div># #</div><div># This script can be run under a wrapper shell script #</div><div># and then used (if desired) to checkin the firewall #</div><div># configurations to SVN or any other change managment #</div><div># or archiving solution - I will post my wrapper once #</div><div># properly sanitized. #</div><div>#######################################################</div><div>#</div><div>##set an overall timeout if individual commands hang</div><div>set timeout 15</div><div>#</div><div>##set the directory you'd like the script to work out of</div><div>set -- /home/user/*</div><div>#</div><div>##open a pre-built list of IPs/FQDN hostnames delimited by line returns</div><div>##Note: it may be necessary to SSH to each host manually as serviceaccountname </div><div>##to save SSH host key</div><div>set f [open "/home/user/firewall_list"]</div><div>#</div><div>##for each entry in firewall_list, SSH to host and move to enable</div><div>##Note: enable may not be required if you customize the permissions </div><div>##on the serviceaccountname</div><div>while {[gets $f line] != -1} {</div><div> set pw1 "password_1"</div><div> set pw2 "password_2"</div><div> spawn "/bin/bash"</div><div> send "ssh serviceaccountname@$line\r"</div><div> expect "\npassword:"</div><div> send "$pw1\r"</div><div> expect -re "(\[^\r]*)>" {</div><div> set name $expect_out(1,string)</div><div> }</div><div> send "enable\r"</div><div> expect "\nPassword:"</div><div> send "$pw2\r"</div><div> expect "#"</div><div>##determine whether this is an FWSM or an ASA and appropriately save </div><div>##running-config to disk</div><div> send "sho version\r"</div><div> expect {</div><div> "FWSM" {</div><div> send \x20</div><div> set devtype "FWSM"</div><div> expect "#"</div><div> send "copy /noconfirm running-config disk:/$name\r"</div><div> }</div><div> "Adaptive Security Appliance" {</div><div> send \x20</div><div> set devtype "ASA"</div><div> expect "#"</div><div> send "copy /noconfirm running-config disk0:/$name\r"</div><div> }</div><div> }</div><div> expect "#"</div><div> sleep 2</div><div> send "logout\r"</div><div>#</div><div>##open a new bash shell and SCP the config that had been saved to disk </div><div>##back to the host this script is running on</div><div> spawn "/bin/bash"</div><div> send "scp serviceaccountname@$line:$name /home/user/workdir/$name\r"</div><div> expect -re ".*password:.*"</div><div> send "$pw1\r"</div><div> expect "100%"</div><div> sleep 1</div><div> }</div><div>#</div><div><span><span></span></span></div></div>Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com2tag:blogger.com,1999:blog-443223739594249029.post-84363878350425060392010-07-23T13:29:00.000-07:002010-08-02T11:35:02.443-07:00Old BBQ = water rocket launcher!<div><br /></div><div><br /></div>P.S. I'll be adding more build pics as I upload them....<br /><div style="text-align: center;"><br /></div><div style="text-align: center;"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh45XRi7Ju9z09DUwri7RMA-s_lBWKb9oTdH5dKcRoRnHIBAarwThqazHQZOFYnoqKp3-nIwYxoNj28DC7M8v4AyUANGKaoY9Z1llK8ARS7FJXbhDVKQ4DbZY2s1FGv6MzObEdR7gkubVg/s1600/PICT0028.JPG"><img style="cursor: pointer; width: 400px; height: 300px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh45XRi7Ju9z09DUwri7RMA-s_lBWKb9oTdH5dKcRoRnHIBAarwThqazHQZOFYnoqKp3-nIwYxoNj28DC7M8v4AyUANGKaoY9Z1llK8ARS7FJXbhDVKQ4DbZY2s1FGv6MzObEdR7gkubVg/s400/PICT0028.JPG" alt="" id="BLOGGER_PHOTO_ID_5497234162292329570" border="0" /></a></div><div><div style="text-align: center;"><br /><span><span></span></span></div></div><br /><div style="text-align: center;"><object height="344" width="425"><param name="movie" value="http://www.youtube.com/v/se0-w23iLUE&hl=en&fs=1"><param name="allowFullScreen" value="true"><param name="allowscriptaccess" value="always"><embed src="http://www.youtube.com/v/se0-w23iLUE&hl=en&fs=1" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" height="344" width="425"></embed></object><br /></div><div><br /></div>So a friend was having a "water rocket party" and we, as dads, were going to build water rocket launchers. It wasn't a competition, per se, but rather a way to have enough launchers that even if one or two failed, every kid would have a chance to launch their rocket.<br /><br />His idea was to take a bunch of 2 litre bottles and spray paint them white, then allow the kids to use Sharpie markers, duct tape and other craft items to customize (decorate) their rockets. Great fun, because every kid has a chance to do their own thing and yet have fun with the other kids, and a good chance of getting really wet when we launched the rockets.<br /><br />During my design phase, I researched a bunch of launchers on the internet. Trust me when I tell you that there are alot of different designs to choose from. They all were practical and easy to build, but they lacked 2 things:<br /><br />1. They were not built very "heavy duty". They were built to be used a few times and then discarded or rebuilt.<br />2. They were slow to load. Most of the designs required pouring the water into the bottle prior to being put on the launcher, then required connecting the compressed air and pressurizing.<br /><br />My problem was that kids wanted to enjoy this, so it had to handle a certain amount of abuse. It had to be quick to load, because kids aren't typically known for having a great deal of patience.<br /><br />When I was in high school shop class, we built this monstrosity of a water rocket launcher, with a welded steel frame and plumbed water and air feeds. It was great for a launching alot of rockets quickly and certainly was tough, but I didn't have the resources to build something like that in my garage.<br /><br />My solution?<br /><br />Retrofit my old BBQ to be used as a water rocket launcher! I'd use the BBQ as a frame and then use common hardware store items to create a "simple to use" water rocket launcher.<br /><br />First, I separated the BBQ from the rest of the BBQ parts. Then, I used a drill and bolts to connect one square pipe running end to end.<br /><br />Next, I confirmed that an outer square pipe fit around that first square pipe. This second pipe (they call it C channel?) would be my sliding mechanism and eventually act as a release for the bottle.<br /><br />Next, I drilled a hole in the inner square pipe and installed a T fitting upside down. One input would be water, one would be air and the output (pointed up) would be the bottle. When building this T piece, I took a 2L bottle with me to the hardware store (in my case Lowes) to make sure what I was building actually fit. This would now act as my launch mount.<br /><br />I bolted the launch mount loosely in place. At the hardware store, I purchased 2 ball valves, for controlling the input of both the water and the air, as well as some air hose (which would handle the necessary pressure) and connect my inputs to my launch mount.<br /><br />Adding a bit more control, I installed a pressure gauge just beyond the input air valve so that I could measure the air pressure I'd be adding to the system.<br /><br />Placing the bottle on my launch mount, I took an old metal tool and trimmed it with a hacksaw to act as a fork that would be able to hold and lock the bottle onto the launch mount. Using a carefully crafted piece of 2X4 wood, I was able to attach the fork onto the sliding mechanism using wire ties (AKA zip ties). I now had a working lock and release and decent inputs.<br /><br />A few wire ties later (to attach the ball valve inputs to the frame) and I had a working mock up that I could actually demonstrate.<br /><br /><br /><br />Please note: If you attempt anything like this own, you do so at your own risk. For example, the pressures involved can be unsafe for the 2L bottle, not to mention if you accidentally pressurize your home's water system. Understand the risks before proceeding with anything.<br /><br />After I tested the mockup, I decided to change my launch mount so that it withstood slightly more pressure. After a trip to Lowes & a helpful employee named Vern, I used a few additional PVC fittings and a stack of rubber O rings and was able to get a pretty decent seal on the bottle. This required me using a taller 2X4 wood piece.<br /><br />In the end, I tore everything down and painted the frame with some good Rustoleum paint (hammerlite finish to hide the imperfections). Putting it back together, I used wire ties to secure the air hose to the frame and used the right size of hose clamps to hold the input valves to the frame, the fork to the sliding mechanism, and each air hose on its respective fitting. Two more hose clamps to connect a handle to the sliding mechanism and I was ready! And I didn't forget to leave an air compressor quick connect so that we could easily connect and disconnect the compressor.<br /><div style="text-align: center;"><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPoIDEDF2KDt72Lrrn3GR9aGBckL6LSH_Hzgi4LmCdU0ddULGtxVhSozYiIDRYVs1U-uV86ebDKp56As_zkcC3qeusZPXXL22jodbOhRhKSd5mn75QjdZ-9FFwWHhTMDmO5aQnKOCDR18/s1600/PICT0029.JPG"><img style="cursor: pointer; width: 400px; height: 300px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPoIDEDF2KDt72Lrrn3GR9aGBckL6LSH_Hzgi4LmCdU0ddULGtxVhSozYiIDRYVs1U-uV86ebDKp56As_zkcC3qeusZPXXL22jodbOhRhKSd5mn75QjdZ-9FFwWHhTMDmO5aQnKOCDR18/s400/PICT0029.JPG" alt="" id="BLOGGER_PHOTO_ID_5497234698467089362" border="0" /></a><br /><div style="text-align: center;"><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0b62rxKe3ECQkdXAdjOWiBhB0gmtRu7W6bD-k4MSYv5leUNvX-PcAUAw8JNPw6UGZwKw0VTw1j84YWCFH4Z6wg_3ikNHgjLDLwabFydujBfiIk3JuxaMIoIsCbMHhYRRMim5On7cLNh8/s1600/PICT0032.JPG"><img style="cursor: pointer; width: 400px; height: 300px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0b62rxKe3ECQkdXAdjOWiBhB0gmtRu7W6bD-k4MSYv5leUNvX-PcAUAw8JNPw6UGZwKw0VTw1j84YWCFH4Z6wg_3ikNHgjLDLwabFydujBfiIk3JuxaMIoIsCbMHhYRRMim5On7cLNh8/s400/PICT0032.JPG" alt="" id="BLOGGER_PHOTO_ID_5497235658743788962" border="0" /></a><br /></div><div style="text-align: center;"><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1QL1tscxaxN3EUowaYM0jOfdnWjtu5fhvGGmKD-ZS6HXdicWJ6tB_ZCaoq0Y1WK6gUjQqp_5ftPXyeLEpoQ703q9YLRKT9WWeNhP7T38Yxb4gwyikYQJqLAUEerDMLpaGMcujT2Oqp7A/s1600/PICT0033.JPG"><img style="cursor: pointer; width: 400px; height: 300px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1QL1tscxaxN3EUowaYM0jOfdnWjtu5fhvGGmKD-ZS6HXdicWJ6tB_ZCaoq0Y1WK6gUjQqp_5ftPXyeLEpoQ703q9YLRKT9WWeNhP7T38Yxb4gwyikYQJqLAUEerDMLpaGMcujT2Oqp7A/s400/PICT0033.JPG" alt="" id="BLOGGER_PHOTO_ID_5497236181279892802" border="0" /></a><br /><br />Shameless plug: Tire repair system that I think rocks:<br /><h2><a href="http://www.ventura-mca.com/gryypvideo.aspx">Gryyp Emergency Puncture Tire Repair Systems</a></h2><br /></div><br /><span><span></span></span><span><span></span></span><span><span></span></span></div>Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com2tag:blogger.com,1999:blog-443223739594249029.post-80752308628403337722010-06-14T12:34:00.000-07:002010-06-14T12:45:37.842-07:00ARM Toolchain for libspmp8000<span style="font-family:verdana;">Here is a toolchain builder that I edited from </span>http://www.gnuarm.com that will help people build an ARM toolchain to start building programs for the spmp8000 series devices like the JXD1000 JXD2000 and any of those types of devices made by Suncom. This is designed on and built for Ubuntu 10.04 LTS but should work on newer versions of Fedora (sorry...haven't tested that yet).<br /><br />All directions are found within the script.<br /><br />#!/bin/sh<br /><br />## Script largely based on gnu-arm-installer but customized for building a toolchain for the smpm8000<br />## script customized by Paklids June 14 2010<br />## *** for future be aware of the versions of source archives used (like gcc and binutils) and try to match those as closely as possible to the versions running natively on your build machine ***<br /><br />## Prerequisites<br />## download and install these to your distribution (based on Ubuntu for now)<br />## 1. apt-get install build-essential<br />## 2. apt-get install libpng3 libpng12-dev libncurses5-dev<br />## 3. apt-get install subversion<br />## (hope there isn't enything else I'm missing...)<br /><br /><br />## Steps to build toolchain<br />## 1. create a parent directory for your toolchain (using mkdir) and cd into it<br />## 2. create 3 subdirectories 'src' 'build' & 'install'<br />## 3. Download these source archives into the 'src' subdirectory but don't unpack<br />## ftp://ftp.gnu.org/gnu/gcc/gcc-4.4.3/gcc-4.4.3.tar.bz2<br />## ftp://ftp.gnu.org/gnu/binutils/binutils-2.20.1.tar.bz2<br />## ftp://sources.redhat.com/pub/newlib/newlib-1.18.0.tar.gz<br />## ftp://sourceware.org/pub/insight/releases/insight-6.8.tar.bz2<br />## 4. move this script to the toolchain parent directory & make this script executable (using something like chmod u+x ./custom-arm-toolchain)<br />## 5. Launch script & let it run for a long while<br />## 6. Make sure the ?/parent-dir/install/bin is in your $PATH. do an 'echo $PATH', then try an 'export PATH=/fullpathto/parent-dir/install/bin:$PATH', then test again using 'echo $PATH'<br />## 7. After this you should be able to issue the command 'arm-elf-gcc' and it should NOT complain about 'command not found'.<br /><br /><br />ROOT=`pwd`<br />SRCDIR=$ROOT/src<br />BUILDDIR=$ROOT/build<br />PREFIX=$ROOT/install<br /><br />GCC_SRC=gcc-4.4.3.tar.bz2<br />GCC_VERSION=4.4.3<br />GCC_DIR=gcc-$GCC_VERSION<br /><br />BINUTILS_SRC=binutils-2.20.1.tar.bz2<br />BINUTILS_VERSION=2.20.1<br />BINUTILS_DIR=binutils-$BINUTILS_VERSION<br /><br />NEWLIB_SRC=newlib-1.18.0.tar.gz<br />NEWLIB_VERSION=1.18.0<br />NEWLIB_DIR=newlib-$NEWLIB_VERSION<br /><br />INSIGHT_SRC=insight-6.8.tar.bz2<br />INSIGHT_VERSION=6.8<br />INSIGHT_DIR=insight-$INSIGHT_VERSION<br /><br />echo "I will build an arm-elf cross-compiler:<br /><br /> Prefix: $PREFIX<br /> Sources: $SRCDIR<br /> Build files: $BUILDDIR<br /><br />Press ^C now if you do NOT want to do this."<br />read IGNORE<br /><br />#<br /># Helper functions.<br />#<br />unpack_source()<br />{<br />(<br /> cd $SRCDIR<br /> ARCHIVE_SUFFIX=${1##*.}<br /> if [ "$ARCHIVE_SUFFIX" = "gz" ]; then<br /> tar zxvf $1<br /> elif [ "$ARCHIVE_SUFFIX" = "bz2" ]; then<br /> tar jxvf $1<br /> else<br /> echo "Unknown archive format for $1"<br /> exit 1<br /> fi<br />)<br />}<br /><br /># Create all the directories we need.<br />#mkdir -p $SRCDIR $BUILDDIR $PREFIX<br /><br />(<br />cd $SRCDIR<br /><br /># Unpack the sources.<br />unpack_source $(basename $GCC_SRC)<br />unpack_source $(basename $BINUTILS_SRC)<br />unpack_source $(basename $NEWLIB_SRC)<br />unpack_source $(basename $INSIGHT_SRC)<br />)<br /><br /># Set the PATH to include the binaries we're going to build.<br />OLD_PATH=$PATH<br />export PATH=$PREFIX/bin:$PATH<br /><br />#<br /># Stage 1: Build binutils<br />#<br />(<br />(<br /># autoconf check.<br />cd $SRCDIR/$BINUTILS_DIR<br /><br />) || exit 1<br /><br /># Now, build it.<br />mkdir -p $BUILDDIR/$BINUTILS_DIR<br />cd $BUILDDIR/$BINUTILS_DIR<br /><br />$SRCDIR/$BINUTILS_DIR/configure --target=arm-elf --prefix=$PREFIX --disable-werror && make all install<br /><br />) || exit 1<br /><br />#<br /># Stage 2: Patch the GCC multilib rules, then build the gcc compiler only<br />#<br />(<br />MULTILIB_CONFIG=$SRCDIR/$GCC_DIR/gcc/config/arm/t-arm-elf<br /><br />echo "<br /><br />MULTILIB_OPTIONS += mno-thumb-interwork/mthumb-interwork<br />MULTILIB_DIRNAMES += normal interwork<br /><br />" >> $MULTILIB_CONFIG<br /><br />mkdir -p $BUILDDIR/$GCC_DIR<br />cd $BUILDDIR/$GCC_DIR<br /><br />$SRCDIR/$GCC_DIR/configure --target=arm-elf --prefix=$PREFIX \<br /> --enable-languages="c" --with-gnu-as --with-gnu-ld --with-cpu=arm926ej-s \<br /> --disable-nls --disable-werror --disable-libssp --disable-libgomp --disable-libmudflap --disable-werror && make all-gcc install-gcc<br /><br />) || exit 1<br /><br />#<br /># Stage 3: Build and install newlib<br />#<br />(<br />(<br /># Same issue, we have to patch to support makeinfo >= 4.11.<br />cd $SRCDIR/$NEWLIB_DIR<br /><br />) || exit 1<br /><br /># And now we can build it.<br />mkdir -p $BUILDDIR/$NEWLIB_DIR<br />cd $BUILDDIR/$NEWLIB_DIR<br /><br />$SRCDIR/$NEWLIB_DIR/configure --target=arm-elf --prefix=$PREFIX --disable-newlib-supplied-syscalls --disable-werror && make all install<br /><br />) || exit 1<br /><br />#<br /># Stage 4: Build and install the rest of GCC.<br />#<br />(<br />cd $BUILDDIR/$GCC_DIR<br />echo "*** Re-CONFIGUREing GCC ***"<br />$SRCDIR/$GCC_DIR/configure --target=arm-elf --prefix=$PREFIX \<br /> --enable-languages="c,c++" --with-gnu-as --with-gnu-ld --with-cpu=arm926ej-s \<br /> --disable-nls --disable-werror --disable-libssp --disable-libgomp \<br /> --disable-libmudflap --with-newlib --with-headers=$SRCDIR/$NEWLIB_DIR/newlib/libc/include<br />sleep 60<br />echo "*** Re-MAKEing GCC ***"<br />make<br />make install<br />sleep 60<br /><br />) || exit 1<br /><br />#<br /># Stage 5: Build and install INSIGHT.<br />#<br /><br />(<br /># Now, build it.<br />mkdir -p $BUILDDIR/$INSIGHT_DIR<br />cd $BUILDDIR/$INSIGHT_DIR<br /><br />$SRCDIR/$INSIGHT_DIR/configure --target=arm-elf --prefix=$PREFIX --disable-werror && make all install<br /><br />) || exit 1<br /><br /><br />export PATH=$OLD_PATH<br /><br />echo "<br />Build complete! Add $PREFIX/bin to your PATH to make arm-elf-gcc and friends<br />accessible directly.<br />"Paklidshttp://www.blogger.com/profile/08394369209628528940noreply@blogger.com1