I wrote about effective learning in 2022. In 2023 I completed two MOOC courses: Learning How to Learn and Mindshift, though which I gained a deeper understanding of effective learning. This blog is a continuum from the last piece. The...
productivity
effective_learning
Kafka Consumer Concepts Consumers and Consumer Groups The main way we scale data consumption from a Kafka topic is by adding more consumers to a consumer group - adding consumers in a consumer groups scales up the process, however adding...
reading_notes
Kafka
Why My GitHub Page Failed to Deploy I was working on a few tech blogs on my reading notes. I felt that the content of the blogs became verbose, it would be nice to add a table of content to...
ci/cd
github_actions
Kafka Producer Components Illustrated Basic Setup Properties kafkaProps = new Properties(); // <= start with a property object kafkaProps.put("bootstrap.servers", "broker1:9092,broker2:9092"); // <= server uri kafkaProps.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer"); // <= use default serializer kafkaProps.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); producer = new KafkaProducer<String, String>(kafkaProps); Sending Messages...
reading_notes
Kafka
Hands-on Running on AWS EC2 (ubuntu, t2.micro) ssh to server ssh -i yulu-mac.pem ubuntu@xx.xx.xx.xx install java 11 sudo apt-get update sudo apt install openjdk-11-jre-headless install zookeeper wget https://dlcdn.apache.org/zookeeper/zookeeper-3.5.10/apache-zookeeper-3.5.10-bin.tar.gz tar -zxf apache-zookeeper-3.5.10-bin.tar.gz mv apache-zookeeper-3.5.10-bin /usr/local/zookeeper mkdir -p /var/lib/zookeeper cp > /usr/local/zookeeper/conf/zoo.cfg...
reading_notes
Kafka
What Problems does Kafka Solve How Kafka comes in to save the data transportation problems here Apache Kafka was developed as a publish/subscribe messaging system designed to solve the above problem - a distributing streaming platform. A filesystem or database...
reading_notes
Kafka
Chapter Summary Database partition or sharding is discussed in this chapter. Partitioning is a basic concept in data storage engine. When the data volume goes up, a single node is not sufficient to host all the data, we have to...
reading_notes
system_design
Chapter Summary This chapter is the start of the three related topics regarding storage systems, namely replication, partition and transaction. Replication is important because of these these properties it provides: 1) keep the data geographically close to the users 2)...
reading_notes
system_design
Chapter Summary Services communicate to each other by protocols (pre-defined interfaces and data models). As application developers, we are often making API design choices by choosing between HTTP or gPRC, or REST vs RPC vs GraphQL, or Web socket vs...
reading_notes
system_design
Chapter Summary This is an interesting and exciting chapter! Finally the book starts to talk about data systems. The discussion begins with the illustration of how data store can be designed using a simple text file. Extending from the concept,...
reading_notes
system_design
Chapter Summary This chapter briefly discusses three types of data models: relational, document and graph database. Some histories about database are provided to give us a better ideas why these databases are developed and what types of problems they solve....
reading_notes
system_design
The book ‘Design Data Intensive Application’ is a really popular book recently years. Why? The reason is that it is relatively ‘easy’ for the inexperience application software engineers to follow and get a glance of what distributed world looks like....
reading_notes
system_design
There’s an old Chinese saying: sharpening your axe will not delay your job of chopping wood. I didn’t have a clear idea about effective learning until I encountered some popular YouTube videos posted by a YouTuber. I realized how ineffective...
productivity
effective_learning
In order to modularize the ViSearch Android SDK that I was working with previously, I developed my very first UI lib - image-crop-layout. This is a light-weight image cropping module that supports UI customization and box update listener. Here is...
android
jcenter
ui-widget
Start to learn frontend web development. Book and MOOC of March Bought the most recommanded books on Amazon. It is well structured and with really good design. I believe these two books are more for designers than developers. It grasps...
Few months away from my little blog, I decide to make the “I am back post” by going through an interesting brain teaser again. Months ago, one of my closest friends sent a little quiz in our chat group. I...
brain_teazer
TapiWallet is an Android App for e-receipt services provided by TapiData Inc, a startup company incubated in NUS. As exectutive app developer for the project, I took responsibility of all the mobile-related product design and development practices. TapiData provide a...
As a Brain teazers lover, code sometimes can be a fun toy for me. From last year, I started to play around with python challenge occasionally and reached level 13 (with cheating). The famous python challenge seems quite difficult for...
brain_teazer
Occasionally I receives emails asking how to render Camera Frame using OpenGL ES on Android. I lazily used some codes from an open source project InstaCam without fully understand until recently I reviewed some fundamentals of OpenGL ES and re-impelemented...
OpenGL
Accidently found a YouTube video of Dr David Chen’s defence presentation when searching for his paper (and realized I followed his tutorial to study server-client image processing on Android a year ago). It is such an excellent presentation with very...
research
Need to implement some code in Cpp for efficiency, but still want to use Python to rapidly develop the software. Test the boost.python lib to convert Cpp functions to python module. Take reference example from boost.python simple example and the...
This is the first time I practice a methodological design process for Android app. I did not have any professional training on graphic/UI/UX design, just did every step with intuition. Here I note down the process. Random Thoughts on Paper...
It has been a year! A year ago I put up some posts on image processing using Shaders just for my own record. I have never expected that this to be rewarded. Now the work is integrated in my very...
OpenGL
Recently I received some emails asking about the implementation of Lazy Snapping (through a video post on my YouTube). I thought of taking notes of this work, however postponed many times due to lack of time. I dig out some...
research
Recently meet an interesting girl who have some common with me, and we made friends naturally. I felt that she is the same type of person as I am, energetic independent and ..dilettante. We share our thoughts about travel, dream...
productivity
Months ago, tech book author Erik Hellman contacted me through the blog and asked me the permission to include some of my blog posts and source codes about shader programming in his new book on Android Programming. Today I received...
Android
Spent sometime to learn Python for setting up a Django web application framework. Summarize some useful tips for Python programming as well as share a modified program for feature-based homography tracking using OpenCV python wrapper (the sample codes come with...
Haven’t been here writing about techqies for such a long time. During the past few months, I had a wonderful backpack trip to Europe, where I met the Taiwanese guy who is writing shaders for Disney Animation - my dream...
WebGL
Just to record the process to draw two cartoon characters for my best friend’s wedding invitation card. 1) Trace the outline in Illustrator 2) Paint in Photoshop 3) Finish up Thanks to my firends’ trust and appreciation, I gradually start...
During the PhD courses, I have devoted most of my research effort in computer vision and GPGPU programming. Thanks for all the great work by scientists and engineers in Computer Science, we now can easily make the complex image processing...
Android
After I published the Mood Camera app to the play store, I have received some, not too many comments - some are from my friends (all of them give me 5 stars, thanks so much guys), some are from the...
Android
Update on July 18, 2013: A simple program based on OpenCV is on my github. Demo program, not guarantee to work perfectly. Feature descriptors are commonly used in lots of computer vision algorithms - object recognition, tracking, image stitching camera...
research
Finally I got the App release to the public. For so many nights, I stayed up programming, and so many times I almost gave up when facing the technical bottleneck. I’ve also met someone who is potentially to be a...
Rationale The inspirations are from some researches [1][2] in Server-client structure for large-scale AR tracking system. What I’m going to explored as illustrated in the figure below, is to build a backend server for my current outdoor tracking system running...
research
Continued with the previous project - image processing using OpenGL ES shaders, I studied and implemented another two interesting filters: artistic painting effect and diffuse glow effect. Artistic Painting Effect The idea is still taken from the popular book Graphics...
OpenGL
I have spent some time to play around with Facebook and Weibo SDK for Android and create an app for photo-sharing to Facebook and Weibo with in-app log in. Here are the steps and tips. The source code is available...
“Why are you being so mellow?” I was quite often, actually I AM still occasionally, being asked on this question. I am a typical introvert, and I was aware of this quite well since I was small. However for a...
Previous works are handling only single image. Now I consider to apply those shader manipulations on live camera feed in, so that I might achieve a “real-time” filter effect. There is a good example on camera frame mapped to OpenGL...
OpenGL
Finish a parttime course in NAFA in fasion figure drawing. These are some of the work I practiced. It is a great pleasure to catch the beauty and draw it out.
I used to spend quite some time working with Photoshop®, to edit photos or do some digital painting. However I became too lazy to PS since I started to use smartphones. Apps like Instagram® seems to be able to do...
OpenGL
Previously I analyzed the situation I was facing - less efficient in doing my “work” after living in a more healthier way. I urged myself to figure out a way to solve the ‘low efficiency’ problem. Then I was in...
productivity
I have taken a very long time to get myself familiar with OpenGL ES Shaders. Finally I made some codes run and got some results that looked nice. The main area I want to explore is to use shaders to...
One month ago, I decided to change my “unhealthy” lifestyle - stayed up to 4 or 5 am and wake up at noon skipping the breakfast, and started routine work only in the afternoon. After a month of adjusting my...
I decided to translate some posts in Chinese to English and keep a copy of those here. Some design logs I felt quite interesting and worth to be shared. Today share the process for design a simple logo. A logo...
Share the process I adopt to draw on iPad. First I’d like to have a rough sketch on the paper. Only pen can provide such flexibility for free drawing. Sketch Then I take a picture of the sketch, and load...
ARNavi is an Android application that helps anyone new to the Faculty of Engineering in NUS to navigate to the different locations in the faculty, and to get acquainted with the various facilities, offices, and places of interest that can...
Chapter 4 - Good Bad Attitude. Paul talks about such an interesting group of people, who amuse themselves by picking locks - breaking into safes. Those geeks seem to be very difficult to be understood, while, yet can be easily...
reading_notes
Android is interesting, flexible and often too flexible to start with. Here I’d like to share a guide for setup a developing and debugging environment for a native android application in eclipse. The copyright is to the original author of...